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air  traffic  was  made  by  simulating  the  air  traffic  environment  in  a  statistically 
balanced  experimental  setting.  Similar  groups  of  air  traffic  controllers  using  both 
the  ARTS  III  and  the  older  system  were  presented  with  identical  traffic  patterns., 

A  Large  number  of  measures  of  system  performance  were  taken.  Differences  lavoring 
|the  ARTS  system  were  found  i.-i  the  most  critical  of  these  measures.  Safety,  and 
were  translated  into  dollar  benefits  resulting  from  an  expected  reduction  in  mid¬ 
air  collisions.  A  comparison  of  these  dollar  benefits  with  the  incremental  costs 
to  install  and  operate  the  new  equipment  favored  its  installation. 
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EXECUTIVE  PRECIS 


An  expcrijnent  using  PAA  *ir  traffic  controller*  in  an  environment 
designed  to  be  typical  of  low  density  airport  terrainals  was  conducted 
at  NAFEC  during  March  1971.  This  settii®  was  used  in  order  to  compare 
the  ARTS  III  level  of  terminal  automation  with  the  present  "manual" 
system.  A  wide  range  of  important  indicators  of  air  traffic  control 
performance  was  used  in  this  coriparison  intended  as  part  of  the 
evaluation  of  the  decision  to  exercise  the  option  to  purchase  the 
remaining  (third)  lot  of  ARTS  III  equipment;  64  installation  sites 
were  specified  in  the  initial  contract,  29  sites  remained  in  the  third 
lot  option.  In  separate  simulated  experiments,  both  certificated  and 
trainee  controllers  were  presented  with  identical  traffic  samples  in  a 
statistically  controlled  environment.  The  important  findings  were: 

1)  There  was  a  statistically  significant  reduction  in 
communications  workloads  due  to  ARTS  III. 

2)  There  ^^as  a  statistically  significant  reduction  in  the  time 
that  IFR  and  VFR  aircraft  were  in  violation  of  mlnionmi  air  spacing 
standards  (conflicts)  due  to  ARTS  III. 

There  was  an  indicated  reduction  ic  conflicts  between  IFR 
aircraft  due  to  ARTS  III. 

3)  There  were  no  reductions  in  the  time  in  system  or  increases 
in  the  number  of  aircraft  handled  that  can  be  claimed  for  ARTS  at  the 
level  of  automation  and  with  the  terminal  geometry  employed  in  this 
study . 

4)  There  were  no  statistically  significant  dlfferencss  between 
the  trainee  and  certificated  controller  groups  in  the  categories  of 
time  in  system  and  numbers  uf  aircraft  handled.  However,  the  trainee 
group  using  ARTS  III  showed  a  statistically  significant  reduction  in  the 
numbers  of  conflicts  involving  IFR  aircraft.  The  measurements  for 
safety  for  the  trainee  controller  using  ARTS  was  generally  comparable 

to  the  certificated  controller  using  ARTS.  The  performance  of  the 
trainee  group  in  separating  IFR  traffic  with  the  manual  system  was 
totally  unsatisfactory,, 

5)  Follow-on  analyses  of  the  conflict  data  were  performed  in  an 
attempt  to  isolate  those  specific  features  of  ARTS  III  automation  that 
contribute  significantly  to  the  observed  reduction  in  conflicts.  These 
analyses  tend  to  show  that  Mode  C  or  altitude  Infomatien  was  an 
important  contributor,  but  this  conclusion  cannot  be  verified  with 

a  high  degree  of  statistical  confidence. 


u 


u 


I%)Xlar  savings  in  operating  costs  were  daisied  for  ARTS  ill  due  to 
redv^.tvocs  in  training  required  for  controller  certification.  Tlicse 
savixgs  are  not  appreciable,,  hoeever,  and  do  not  affect  the  study's 
resu|ii:s  or  conclusions.  .No  dollar  savings  in  costs. were  claimed  dtie 
to  inductions  in  cooEiunications  workload. 

The.  dollar  benefits  likely  to  result  from  a  reduction  in  midair 
col  ;.isions,  based  upon  the  study's  findings  of  statistically  significant 
irences  in  the  time  in  conflict  for  IFR  x  VFR  aircraft,  were 
e.s.rimated  for  a  wide  range  of  expected  nusdier  of  sddair  collisions  and 
fVir  a  similar  range  of  values  for  the  cost  of  an  accident. 

The  study  concludes  that  the  installation  of  ARTS  III  equipment 
is  justified  for  all  locations  involved  in  the  third  lot  procurement 
the  basis  of  using  conservative  estimates,  the  low  end  of  the  range, 
for  the  number  and  dollar  costs  of  midair  collisions  that  are  expected 
to  occur  in  the  next  decade. 
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SPMMjVRY 


An  experiE!ei>t  employing  air  traffic  controllers  from  the  Federal 
Aviation  Administration  in  a  realistic,  but  simulated,  terminal  environ- 
c^nt  was  conducted  at  the  Rational  Aviation  Facilities  Esper' -Dental 
Center  (KAFEC)  in  Atlantic  City,  New  Jersey,  during  March  1971. 
purpose  was  to  aid  in  deciding  whether  an  option  to  buy  2‘>  syi  terns  c-i 
automated  equipment  —  designated  as  ARTS  III  —  should  be  exercised. 

A  contract  existed  with  the  Univac  Corporation  to  purchase  a  ti  ^al  of 
64  ARTS  111  systems  under  three  separate  option  agreements  for  Installa¬ 
tion  at  major  air  terminals.  The  options  to  buy  lots  one  and  two,  to 
be  installed  at  the  busier  terminals,  had  already  been  exercised. 

A  simple  terminal  geometry,  typical  of  the  less  busy  terminal 
locations  included  in  the  third  lot  option,  was  devised  as  the  appro¬ 
priate  setting  in  which  to  compare  the  ARTS  III  level  of  automation  with 
the  present,  so-called  '‘manual"  system.  In  isolated  experiments, 
certificated  and  trainee  controllers  were  presented  with  identical 
traffic  samples  under  statistically  controlled  conditions.  Detailed 
measurements  oi  comparative  system  performance  —  some  80  specific 
indices  in  seven  major  categories  —  were  taken.  All  measurements 
taken  during  the  run  of  the  experimenc  are  shown  in  Section  I  of  this 
report,  unedited,  along  with  estimates  of  those  statistical  measures 
thought  to  be  most  relevant  to  the  evaluation  of  significant  differences 
between  systems.  The  statistical  analyses  of  these  indices  of 
performance,  designated  to  be  measured  prior  to  the  running  of  the 
experiment  revealed  the  following  general  results; 

/ 


1)  There  was  a  scatisticaliy  significant  reduction,  due  to 
ARTS  III,  in  the  performance  category  of  Communications  Workload. 

2)  There  was  a  sigi  if leant  improvemert  in  the  category  of  Safety, 
due  to  iwvIS  III,  as  measured  by  the  violations  of  minloimi  spacing 
standards  (conflicts) .  Conflicts  between  aircraft  receiv  Lng  navigational 
assistance  as  well  as  a  traffic  advisory  service  from  the  FAA  (those 
operating  under  Instrument  flight  Rules,  IFR)  interacting  with  air¬ 
craft  not  under  FAA  control  (those  operating  under  Visual  Flight 

Rul^s,  VFR)  were  reduced  to  a  statistically  significant  degree. 

Conflicts  between  IFR  aircraft  were  reduced  to  an  indicated,  though 
not  statistically  significant,  degree  using  ARTS  III. 

3)  There  was  no  reduction  in  the  category  of  Time  in  the  Terminal 
Ai.ea,  and  no  increase  in  the  Numbers  of  Aircraft  Handled  demonstrated 
by  ARTS  equipment  at  the  level  of  automation  and  for  the  terminal 
geometry  employed  in  this  study. 

4)  There  were  no  differences  betv*een  the  trainee  and  v.iitificated 
controller  groups  in  the  categories  described  above  in  item  3:  Time 

in  the  Terminal  Area;  Numbers  of  Aircraft  Handled.  The  total 
performance  of  the  trainee  group,  using  ARTS  III  equipment,  was  generally 
comparable  to  that  of  the  certificated  group.  However,  the  ability  of 
the  trainee  controller  to  avoid  conflicts  was  highly  unsatisfactory 
when  using  the  present  manual  system  of  control.  This  latter  finding 
indicated  the  possibility  for  aeriving  dollar  benefits  from  the  reduction 
in  the  period  of  training  required  for  controller  certification.  This 


possibility,  uncovered  in  an  experinent  using  simulated  conditions, 
has  been  verified  by  recent  accounts  of  the  experience  in  training 
controllers  in  the  use  of  ARTS  equipment  reported  by  the  Atlanta 
terminal  facility. 

The  general  theme  suggested  by  the  above  findings  is  that  ARTS  III 
equipment  provides  improved  system  performance  in  the  most  critical  of 
areas-  safety.  Based  upon  these  experimental  findings  it  was  postu¬ 
lated  that  a  major  factor  contributing  to  the  ability  to  control 
traffic  safely  with  fewer  conflicts  is  the  increased  availability  of 
information  in  three  dimensions  concerning  the  location  of  all  aircraft 
within  the  terminal  area.  Aircraft  for  which  spatial  information  in 
the  X,  y  and  z  coordinates  is  available  are  defined  as  "known"  to  the 
system.  All  IFR  aircraft  meet  this  definition.  The  radar  contact  pro¬ 
vides  spatial  information  to  the  ground  controller  in  two  dimensions. 
Radio  communication  with  the  pilot  confirms  the  third  dimension,  the 
aircraft's  altitude.  Therefore,  under  the  present:  manual  system, 
information  in  three  dimensions  is  as  recent  as  the  last  radio  contact. 
For  the  ARTS  system,  this  information  is  updated  to  be  as  recent  as  the 
last  radar  sweep,  if  the  aircraft  is  equipped  with  a  beacon  transponcier 
with  Mode  C  capability  (i.e.,  the  capability  to  transmit  altitude 
information  automatically,  in  response  to  an  interrogation  by  a  ground 
based  radar  beacon) .  More  importantly,  according  to  the  definition 
adopted  —  aircraft  for  which  spatial  information  in  all  three  dimensions 
is  lacking  are  i.  jgarded  as  "unknowns"  —  all  VFR  aircraft  are  unknown 
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to  the  present  system  of  terminal  control,  whereas  that  portion  of  the 
VFR  traffic  equipped  with  Mode  C  transformers  are  known  to  the  ARTS 
system.  It  is  this  latter  feature  of  control  that  the  study  postulates 
to  be  the  discriminating  element  in  the  observed  reduction  in  conflicts, 
although  other  desirable  features  of  control  available  from  the  ARTS 
may  have  contributed  as  well.  An  extensive  analysis  of  the  experimental 
data  was  undertaken  to  investigate  whether  it  was  possible  to 
specifically  identify  the  features  included  in  the  package  of  automation 
available  from  ARTS  III  that  contributed  most  importantly  to  improved 
system  performance.  This  analysis  was  designed  to  answer  the  questions; 
1)  jere  the  data  on  the  target's  ground  speed,  displayed  automatically  on 
the  ARTS  radar  display,  important  to  the  controller;  2)  was  altitude 
information  important ;  and  3)  did  the  mere  "freshness"  of  the  data 
contribute  to  the  controller's  performance?  The  answers  to  these 
questions  would  help  immeasurably  in  defining  the  requirements  for 
future  installations  of  automated  equipment. 

An  operational  analysis  of  the  data  was  performed,  after  the 

experiment,  by  NAFEC  personnel  experienced  in  the  methods  and  procedures 

« 

of  air  traffic  control.  In  addition,  The  MITRE  Corporation  and  the 
Transportation  Systems  Center  of  the  Department  of  Transportation 
performed  separate  statiscical  analyses  designed  to  isolate  those 
automation  features  which  proved  to  be  most  beneficial  to  the  controller. 
The  results  of  these  follow-on  analyses  are  contained  in  Appendix  A  to 
this  report.  A  summary  description  of  these  post -experiment  analyses 
and  their  findings  follow: 


o 


<} 
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1)  The  operational  analysis  which  was  performed  by  NAFEC  applied 
judgmental  factors  based  upon  experience  in  order  to  weight  the  number 
of  conflicts  by  their  degree  of  severity.  The  result  was  that  conflicts, 
adjusted  for  seriousness  of  the  violation,  were  determined  to  be  less 
severe,  in  both  horizontal  and  vertical  directions,  for  the  NAFEC 
certificated  controller  and  the  trainee  controller  usinr;  ARTS.  But, 

it  was  not  possible  to  specifically  identify  the  automation  feature 
contributing  most  significantly  to  this  result. 

2)  The  MITRE  Corporation  conducted  statistical  regression  analyses 
of  the  experimental  data  designed  to  determine  a  pattern  for  the  number 
of  conflicts  occurring  on  a  controller's  radar  scope  at  any  specified 
instant  of  time.  Sample  measurements  of  conflict  information  were  taken 
at  one  minute  intervals.  The  analysis  of  these  conflict  measurements, 
which  included  a  consideration  of  the  degree  of  seriousness  of  the 
violation,  revealed  that  the  number  of  unknown  targets  on  the  scope  aie 
a  statistically  significant  explanatory  variable  of  the  reduced  numbers 
of  conflicts  observed  for  the  ARTS  system.  This  confirms  the  position 
held  prior  to  the  conduct  of  the  experiment.  However,  the  analysis  also 
revealed  that  the  factors  which  influence  conflicts  are  so  numerous  and 
diverse  that  it  is  not  possible  to  conclude  on  the  basis  of  these  data 
that  any  single  feature  of  ARTS  III,  such  as  Mode  C  or  altitude  capa¬ 
bility,  contributed  in  an  appreciable  way  to  the  observed  reduction  in 
conflicts. 


i 


3)  The  Transportation  Systems  Center  was  assigned  the  task  of 
performing  some  additional,  more  analytically  sophisticated,  statistical 
tests  to  determine  the  contribution  to  improved  safety  made  by  the 
addition  of  airborne  transponders  with  Mode  C  capability.  The  idea 
here  was  to  determine  whether  these  aircraft  which  are  involved  in 
violations  of  airspace  standards  are  affected  significantly  by  the  fact 
that  they  were  equipped  with  airborne  transponders  having  Mode  C  capa¬ 
bility.  This  is  in  contrast  to  the  previous  analysis  which  determined 
that  transponder  equipped  aircraft  reduced  system  conflicts  in  general, 
but  which  did  not  attempt  to  determine  whether  they  were  reduced  for 
those  specific  aircraft  involved  in  the  violation.  Unfortunately,  both 
the  traffic  samples  and  the  design  of  the  experiment  conducted  at  NAFEC 
did  not  allow  for  this  kind  of  extended  statistical  treatment.  The 
only  conclusion  possible  is  that  any  effect  that  transponders  with  Mode  C 
might  have  in  keeping  aircraft  having  this  equipment  out  of  conflicts, 
cannot  be  confirmed  with  high  reliability  by  the  experiment  recently 
conducted.  It  is  hoped  tnat  this  effect  will  be  reinvistigated  in  a  future 
experiment  more  appropriately  designed  to  answer  this  specific  question. 

The  results  of  the  experiment  —  all  the  data  '•ollected,  expressed 
in  th'^ir  physical  units  of  measurement  —  are  shown  in  Section  I  of  the 
study.  An  interpretative  Section  II,  which  assesses  the  validity  of 
applying  these  results  from  a  simulated  experiment  to  real-world 
problems,  follows. 


An  estimate  of  the  dollar  benefits  likely  to  result  from  the 
installation  of  ARTS  equipment  is  shovn  in  Section  III.  Since  it  is 
not  possible  to  remove  all  of  the  subjective  judgment  that  is  necessary 
to  perform  these  analyses,  the  arr^  of  assuBq>tlcns  and  inferences  used 
to  obtain  dollar  estimates  of  the  benefits  to  be  derived  from  ARTS  III 
equipment  are  identified  explicitly  in  this  section  in  order  chat  the 
reader  might,  at  least,  confirm  the  estimates. 

One  of  the  pitfalls  of  cost/benefit  analyses  is  that  dollar  benefits 
are  frequently  double-counted,  and  more.  However,  it  is  usually 
possible,  when  dealing  with  multidimensional  indices,  to  translate 
changes  in  one  dimension  of  performance  into  any  of  the  others.  For 
example,  a  reduction  in  the  safety  dimension  --  conflicts  --  may  be 
translated  into  1)  a  reduction  in  delays  by  asking  the  question;  "For 
the  same  number  of  conflicts,  how  much  longer  would  it  take  aircraft  to 
arrive  or  depart  from  a  terminal  using  a  manual  system  of  control,"  or 
into  2)  an  increase  in  controller  capacity  by  rephrasing  the  question  to 
determine  the  increased  numbers  of  aircraft  that  could  be  handled  for 
any  identical  number  of  conflicts.  Similarly,  the  reductions  in 
communications  workload,  which  was  demonstrated  to  be  significantly 
lower  for  ARTS  III  controllers  at  very  high  levels  of  statistical 
confidence,  could  be  translated  into  a  quantitative  measure  of  the 
probability  that  any  given  communication  channel  would  be  congested. 

This  probability  could  then  be  re-translated  into  a  probability  of 
reduced  conflicts.  But,  it  is  clearly  in  error  to  compute  benefits  by 
summing  up  the  separate  dollar  estimates  for  1)  Safety,  2)  Delays, 


-  3  - 


3)  Capacity,  and  4)  CoagiBa icat Iona  Vorkl^ac  iioen  it  is  evident  tisat 


these  seemingly  different  dimensions  of  perfomance  are  not  naitnaily 


exclusive-  However,  there  is  one  dimension  to  the  job  of  controlling 


aircraft  that  dcsinates  all  the  others.  Safety.  For  this  reason,  the 


dollar  amounts  of  benefits  attributable  to  ASTS  III,  shown  in  Section  ii. 


are  based  primarily  upon  this  single,  but  critical,  disension  of  the  job. 


Dollar  benefits  due  to  increased  safety  were  based  upon  the  observed 


experimental  finding  —  index  54  —  that  there  is  a  statistically 


significant  difference  in  the  time  in  conflict  in  the  tersinai  area 


between  known  aircraft  (IFR)  and  unknown  aircraft  (VFR)  due  to  ASTS  III. 


This  experimental  finding  of  a  significant  reduction  in  the  time  spent 


in  conflict  was  translated  into  a  reduced  probability  of  a  midair 


collision.  Other  experimental  findings,  such  as  statistically  signifi¬ 


cant  reductions  in  a  wide  category  of  measinres  for  ccamin  icat  ions  work¬ 


load  were  not  translated  into  dollar  benefits  for  ARTS  III,  but  were 


assinted  to  be  complementary  influences  contributing  to  the  observed 


reduction  in  Index  54,  the  time  in  conflict.  Therefore,  except  for  a 


small  dollar  benefit  attributed  to  the  experimentally  observed,  and 


empirically  verified,  ability  of  ARTS  III  to  reduce  the  time  required  to 


train  an  apprentice  controller,  all  dollar  benefits  are  based  on  the 


single  job  dimension  of  safety.  Every  attempt  was  made  to  eliminate 


any  redundancy  in  dollar  benefits  claimed  for  ARTS  III. 


The  dollar  amounts  of  benefits  shown  in  Section  II  were  estimated 


for  a  wide  range  of  assump'tions  regarding  the  types  of  aircraft  likely 


m 
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to  be  sssolved  is  a  Trigiair  collisiao,  asS  for  a  sisilar  vi/£e  raage  of 
valoes  assCTWid  far  ttbe  costs  of  these  accidents.  The  rae^  of  gassi- 
biliciss  for  trrfdiafr  accideats  isclsded  a  coesidexatico  of  collisicrss 
betnees  snail  geaecal  ariatice  aircraft  vits  siagle  ccca^rots  eamiog 
lov  f  iBcope-s  to  as  alteraati've  cossideratio:'  of  a  ccllisios  be  twees 
jonbo  jets  foil  of  passengers  eatsieg  bigs  ixscooes. 

The  sto&y  coaclsades  tisat  the  isstallatios  of  ABTS  111  ecmipnest  is 
jcstified  at  all  locatioos  saoed  is  tbe  third  lot  prcc.nrenest  of  ASTS  III 
eres  shen  the  lew  end  of  tlae  range  —  small  aircraft  with  single 
occepants  earsisg  lew  ipcomes  —  are  assssned  to  be  typical  of  tbe 
sidair  collisicss  likely  to  occur  is  tbe  seat  decade. 
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CME3imT5  sysgKs  ams£js.  ix  mKmtL  jygsfts 

Ttegaifgal  cr*ffic  cotssists  cf  *ircr*ft  <n^rmzisg  £n  Ebe  centiral  *raa 
sariter  citber  iasETonenrt  «rr  Tissal  rales  (EF$  cr  ^FS).  *£r  rrafrtc 

cictsrrel  *srisspries  —  agrigarigtsal  assistaacc  pltcs  seaazatLra  serr-Ece  - 
ere  orwrii^si  for  ell  IFS  rrefftc  car  Co  e  sisceace  of  some  4®  jrales, 
cepeneing  co  loceciae. 
i.  Ibe  Xeraei  SrstegE 

CocECrol  of  aircreft  eacerisg  or  cSepsrtiEg  tbe  tetadael  errspeoe 
reoaires  dsec  each  aircraft  be  iaeatified  precisely.  Detailed  iofoma- 
Cica  regarding  aa  eircreft’s  position,  altittsSe  esc  speed  presently 
recaire  sobsteatiei  verfeal  caonnEnicet loss :  1)  coatroller /pilot 

COTtnasicet ioa  is  reqaired  to  apsete  ea  eircreft’s  three-dineasicaai 
position  and  to  respond  to  coatrol  advisories;  2)  tenciaal  coatrollers 
comnnaaicete  vith  ea  roate  controllers  and  aangag  taeaselTes  ia  order  to 
transfer  infonnation  ead  carry  oat  bandoffs  of  control.  For  the  laost 
pert,  teraiasl  controllers  mast  neiateia  a  meatel  picture  of  aircraft 
uader  their  coatroi  fay  observiag  oasarked  target  reflectioas  ca  a  radar 
displa3-,  aided  fay  flight  progress  strips  on  vhicb  pertinent  aircraft 
infonaation  is  recorded.  Tha  preseat  equipaeat  has  ao  capability  to 
display  3  computer  geaerated  data  block  adjacent  to  a  radar  target. 

All  targe-<5  appearing  on  the  radar  scope  isast  be  identified  fay  other 
means.  Liaited  target  identification  is  available,  however,  at  those 
teminals  scheduled  to  receive  ARTS  III  equipaent  in  the  third  lot  buy. 


These  locations  all  have  wide  band  radars  and  beacon  decoding  equipment 
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cajai^le  of  resressstiag  tirsecs  by  «  single  sl*^  if  x  prSansry  r»£*r 
recirm  is  resotirofi,  and  by  «  &rc&l£  5s£r*llarl  siasb  if  z  secaofzry  resnrra 
is  receiTed  fTcnn  ac  zircrzft's  bczcae  Exzssjctafes--  16*  ess&arizi  elasnesc 
is  cbzc  trbe  coTTClzckrs  beraees  cbfc  bfezotits  slzsaes  displzyed  ard  ebe 
zircEzft  czr^s's  wMsziity  ard  zlcircds  ajcsst  be  perfcnrmed  macssxllj  by 
referring  c®  z  set  ®f  flight  progress  strips  stzdced  scjzcesc:  s®  tbe 
radzr  displ^. 


2.  Ihe  jiSSS  Sii  System 

ASTS  '15  is  z  s/steat  ®f  zirbnrre  and  gr>mi:.rig  cqncxcoests,  cstd  in 
oagjgsactiaa  srita  existing  radar  esssipneat,  cajole  of  aatoacsticeliT 
dispiayirg  iaf ornat Lee  recairec  for  tae  coatroi  of  air  traffic,  "ihis 
iafasnnatica  is  displayed  oa  a  coatroiler's  radar  scope  ia  tbe  form  of  a 
block  of  data  t^ged  to  eaca  tzrget.  Toe  airtwrae  ccajpccest  of  tias 
system  is  a  beacoa  traaspoader  that  traasaits  a  signal  ia  respccsse  to  aa 
iaterregat ioa  from  a  groaad  l^sed  radar  beacca.  Ibis  iaterrogatioa  is 
mads  at  each  -^fseep  or  tbe  teraiaai’s  surreiliaace  radar:  a  seocedary 
radar  aateaaa  is  attached  to  the  primary  aateaaa  for  the  p<arpose  of 
transaittiag  and  receiving  these  signals.  The  aircraft's  beacon  response 
is  processed  by  a  gxoaau  coepoaeat  vhich  decodes  tbe  sigael  into  a 
digitized  electronic  iapuise  that  can  be  represented  by  a  ntnaerical 
display  on  a  radar  scope,  A  target  can,  thus,  be  identified  and  tra-.  red 
continually  at  each  sweep  of  the  radar's  antenna;  i.e.,  every  four 
seconds-  The  quantity  and  kinds  cf  information  that  are  displayed 
depend  to  a  large  extent  on  the  amount  c:  iaforoation  that  is  transmitted 
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fey  c&e  xiz’crtiz’s  CTXsssacsSsz.  If  mts  araEScctoSer  is  an  (yiiStiS  type 
zssabls  of  cgassnciccigs  ocIt  6&  (fiscnete  ifig:iC2l  aessases,  cnea  tr&e 
ainilitT  to  cositiTreSy  iiScnri*  ns  aircraft  target  is,  of  cocr», 

Itacicef  to  t&is  ganfoer  of  cjffcs.  'gfaera  is  a  clear  pcssibilirr  of 
aiisii&ntification  of  targets.  Bswerer,  tae  ocaer  and  core  oneralent 
type  of  tras^ttroier  is  not  onij  eosiccec  vies  zi£  £.  ICT  iins^ir£ 

discrete  coic^,  a  total  of  4®96,  bet  tner  are  also  cananle  of  crans-- 
aritcirg  a  signal  aaics  contains  infomarion  regareiag  tne  target's 
altitafe.  A  transpoac^r  afeict  crOTiioes  inf o'raat lots  scfficient  for 

yy^  tTSciing  is  grTa>r»<^  JK  Esartag  Msce  A  casaisility; 
a  transyooier  -sisics  transtnits  as  alcictne  signal  is  cesigsatec  as 
baritg  Kooe  C  canabiiitr.  Ibe  A23S  III  ecsipnest  is  oesigDef,  prinsTilv, 
to  be  sased  with  tae  aesjer  cc<&  traasKssaer  wits  Jfose  C  capability, 
althotgh  tbe  older  type  ct  trasspooaer  witls  64  discrete  codes  caa  be 
accffrnncdated  asc  provided  with  a  redncec  sanber  of  aatontated  features. 

Aa  essential  ccnEXJCsrst  of  tbe  AXIS  ill  systear  is  a  gsouss  based 
ccE^ster-  it  is  tfeis  cocs>cn2ct  that  provi<fe5  tbe  additicoal  levels  of 
asstCEsaticn  that  distisgoisb  tbe  AST3  Hi  system  fron  less  sophisticated 
asd  less  useful  systems  of  control,  beacon  decoding  ecuipment  without 
cesnputer  assistance  is  able  to  display  the  identifying  code  of  an  air¬ 
craft's  transponder  in  numerical  form  only.  But,  this  is  not  the  form 
that  is  used  in  controlling  traffic  or  in  transferring  this  control 
from  the  en  route  center  to  the  terminal,  or  between  terminal  sectors. 
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•gargets  ire  cypicilly  ard  csyaErolSed  by  aa  aip&arsaseric 

dtesngraE  roc.  ?br  erancle,  i  Esitei  Mr  lines  filgSst  mmber  3?-2  is 
presect ly  ^trirtea  as  ca  342  oc  tfee  plastic  strips  or  "scriap  boats"' 
rreiicB  are  mned  aiancsally  across  tse  radar  scope  as  a  fligst  progresses. 
Handorfs  or  trsscsfers  o>f  control  are,  liksaise,  designated  by  an 
aipoacdieric  indicatioo  of  an  aircraft's  fligbt  nanfcer.  less  sophisti¬ 
cated,  Qoh-comprsterized,  systems  are  capable  of  attadiisg  a  snmerical 
data  block  oaiy.  e.g.,  #6349  ratrer  than  Ea  342,  to  a  radar  target. 
Hscerer,  clsis  tag  conld  prove  ccafnsing  to  a  cocstroller  '»iao  mast  con- 
tisnaiiy  make  the  mental  correlation  betaeesi  a  transponder  code  and  a 
fligjit  snmber. 

assisting  zha  controller  in  making  tbe  association,  vltbcct  error, 
betsees  radar  targets  and  tbe  alpbannseric  data  needed  for  tbeir  control. 
Boaever,  once  a  cooputer  is  installed  for  this  purpose,  there  are  addi¬ 
tional  features  that  can  be  derived  from  tbe  ccnputer's  ability  to 
rej«atedly  perform  logical  calculations  that  enhance  the  controller’s 
ability  to  handle  air  traffic.  For  example,  the  controller  can  observe 
a  target  move  along  his  radar  display  and  gauge  its  speed  under  the 
present  system  of  control,  by  experience.  But,  this  task  becomes  more 
difficult  if  many  targets  are  being  worked  and  the  controller's  atten¬ 
tion  is  diverted.  On  the  other  hand,  a  .:omputer  can  record  the  time  it 
takes  an  aircraft  to  move  between  any  two  points  —  the  geography  of  a 
given  terminal  is  stored  in  its  memory  —  and  it  can  display  the  groimd 
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sosz^  of  aa  airtporoe  target  directly  cposa  toe  radar  scape.  Sirnilarly, 
tse  ccmpater  caa  cse  a  stored  progran  of  astfeenatical  alagaritcma  to 
aaticipate  a  target's  fatare  track,  liissed  signals  ox  signals  saich 
are  eqairccal,  doe  to  rerlectioas  or  ghosts,  can  be  resolved  aad  a  data 
block  assigned  to  the  appropriate  tar^t  within  aay  order  of  statistical 
reliability-  Ibe  additional  aatocated  sea  tares  available  from  a  corapater 
software  program  hac  evolved  from  considerable  experience  with  prototype 
installations  at  the  Ifew  York  City  Ccasnon  IFS  Facility  and  at  the  Atlanta, 
Georgia,  facility-  The  antoraated  ecaipment  at  Sew  York,  called  ARTS  lA, 
vzs  ccoanissioned  in  Jane  1969;  the  facility  in  Atlanta,  called  ARTS  I, 

rally  operational  in  September  1966.  The  list  of  ARTS  III  features, 
shown  below  in  Table  1,  are  currently  in  use  at  these  facilities,  but 
both  liew  York  and  Atlanta  reminals  also  have  the  eapability  to  tnack 
targets  based  upon  a  response  received  froo  interrogation  by  the  prinary 
radar,  it  is  expected  that  the  ability  to  track  pritaary  radar  returns 
will  be  added  to  the  present  ARTS  III  equipment  in  follcw-on  updating  of 
this  equipment.  In  addition,  it  is  expected  that  such  further  automated 
features  as  computer  assisted  sequencing  and  spacing  of  arrival  aircraft 
will  also  be  added  to  current  ARTS  III  capability.  The  use  of  ARTS'  III 
as  a  platfora  for  future  levels  of  automation  is  an  ii3q)ortant  element  in 
its  design.  ARTS  III  is  designed  to  be  modular;  increased  computer 
memory  can  be  added  in  discrete,  modular,  units  either  as  traffic  levels 
increase  or  more  automated  features  are  desired.  Despite  these  "platform" 
benefits,  the  present  study  is  limited  to  a  study  of  ARTS  III  for  ARTS  III 
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sake  cslj.  Fo  attesnst  has  bees  mads  to  sisxilete  anj  of  the  follov~on 
features  of  actotnaticsx,  or  to  iroute  benefits  froa  them.  Instead,  the 
study  attempted  to  construct  the  most  realistic  representation  of  the 
follocring  list  of  ASTS  III  features  at  its  MFEC  facility  and  to  limit 
its  appraisal  to  these  features  only. 

Table  I 

MAJCa  FOHCTICSS  (W  ARTS  III 

•  The  automatic  placement  of  data  tag  infomation  -  -  1)  alpha- 
mraeric  identity;  2)  ground  speed;  and  3)  altitude  if  target  ha«; 
Mode  C. 

•  Autonatic  initiation  and  display  for  discrete  coded  transponder 
targets  on  takeoff;  no  keyboard  entry  is  necessary. 

•  Interfacility  handoff  capability  which  transfers  alphanunsric 
tags  between  the  tenainai  and  the  adjacent  en  route  AKTC  Center 
equipped  Stage  A  automation  of  the  National  Airspace  System. 

•  Intrafacility  handoff  capability  which  transfers  tags  from  one 
operating  position  to  another  within  the  terminal. 

•  Quick-look  capability  which  permits  one  position  to  look  at 
alphanumerics  displz^d  at  another  position. 

•  Automatic  track  drop  which  will  eliminate  the  tags  at  a 
predetermined  range  and/or  altitude. 

In  addition,  keyboard  controls  enable  the  controller  to  eliminate 
fields  in  the  tag,  manually  drop  tracks  and  otherwise  tailor  the  physical 
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presentation  (intensity  of  targets  —  alphanusier ics ,  operating  range 
and  offset)  to  his  particular  requireaents. 

#  Flight  emergencies 

A  special  eaergency  code  is  inserted  over  the  alphanuaeric  tag 
of  any  flight  which  develops  an  eaergency.  This  code  appears  on  all 
controller  radar  displays  in  the  facility  and  an  audible  signal  is 
activated  to  alert  all  controllers.  One  control  position  is  then 
assigned  to  work  with  the  distressed  aircraft  until  the  emergency  is 
resolved. 

A  typical  ARTS  III  display  is  shown  in  Figure  1.  The  manual  system 
can  re  described  as  a  similar  display  in  which  no  data  block  is  attached 
to  t  radar  target  shown.  The  writing  of  relevant  information  required 
for  the  control  of  air  traffic  directly  on  the  radar  display  is  the 
essential  element  of  ARTS  level  of  automation.  The  study's  purpose  is 
to  detei -’.'■.ine  whether  the  automijtic  tracking  and  radar  scope  display 
of  this  information  is  sufficiently  beneficial  to  justify  the  installa¬ 
tion  of  ARTS  III  equipment  at  -.hose  terminal  locations  identified  in 
the  third  lot  option, 

ARTS  III  Candidate  locatiocs 

Table  2  identifies  the  terminals  included  in  the  third  lot  option 
to  buy.  The  itinerant  operations  for  FY  1970  at  the  primary  airports 
scheduled  for  ARTS  III  installations  are  also  shown  in  this  table.  These 
numbers  of  operations  were  a  major  part  of  the  criteria  uued  to  select 
the  ARTS  III  candidate  locations.  However,  as  a  result  of  this  study, 
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TYPICAL  ARTS  III  DISPLAY 


Table  2 


AIRCRAFT  OPERATIONS  AT  AIRPORTS 
PROPOSED  FOR  THE  THIRD  LOT  PROCUSIMENT,  ARTS  III 

FY  1970 


(In  iliousands) 


Itinerant  Operations* 

i 

Airport 

Air 

Carrier 

General 

Aviation 

Sum,  i 

Itinerant  | 

Total 

Tampa 

83.0 

61.7 

0.7 

145.4 

12.8 

158.2 

Baltimore 

131.2 

93.5 

2.4 

227.1 

11.4 

238.5 

Portland,  Ore^.on 

91.5 

59.2 

14.5 

165.2 

17.3 

182.5 

Orlando 

— 

136.9 

2.0 

138.9 

57.7 

196.6 

Dayton 

59.5 

1 

97.8 

0.3 

157.6 

1 

44.6 

Omaha 

46.8 

95.0 

1.6 

143.3 

90.1 

233.4 

Nashville 

63.9 

86.2 

7.6 

157.7 

51.1 

208.8 

Jacksonville 

49.1 

30.6  ^ 

11.2 

90.9 

13.1 

104.0 

Louisville  (Stand) 

83.3 

41.3  ! 

8.0 

132.6 

14.0 

146.6 

Birmingham 

48.1 

87.4  j 

17.1 

152.6 

78.9 

231.5 

Hartford  (Windsor  Locks) 

60.2 

66.5 

7.1 

133.8 

21.7 

155.5 

Salt  Lake  City 

68.1 

125.5 

9.9 

203.5 

89.9 

293.4 

Rochester,  N.Y. 

61.3 

69.5 

1  3 

132.0 

102.4 

234.4 

Syracuse 

61.3 

49.8 

9.7 

120.8 

29.9 

150.7 

Tulsa  (Int'l) 

48.5 

109.8 

7.0 

165.4 

45.2 

210.6 

Albuquerque 

46.1 

98.4 

29.6 

174.1 

39.2 

213.2 

Providence,  R.I, 

54.8 

71.7 

9.6 

136.1 

65.3 

201.5 

El  Paso 

38.2 

95.3 

22.3 

155.9 

84.1 

239.9 

Albany 

50.1 

59.1 

1.9 

111.1 

35.1 

146.2 

Tucson 

36.8 

65.9 

10.6 

113.3 

88.0 

201.3 

Shreveport  (Greater) 

36.9 

;  23.0 

0.6 

60.5 

14.8 

75.2 

Charlotte,  N.C. 

62.7 

86.8 

4.4 

1  153.9 

11.0 

164.9 

Burbank 

1  31.2 

147.9 

2.1 

!  181.3 

65.9 

247.1 

Buffalo 

NAFEC  (RdlD)  '  ' 

82.1 

55.5 

1.1 

1  138.7 

1 

1 

30.9 

i 

169.6 

Raleigh/Durham 

32.1 

71.3 

5.9 

109.3 

38.2 

1  1V7.5 

Sacramento  (Metro) 

34,9 

34.7 

1.5 

71.1 

48.0 

119.1 

Des  Moines 

35.3 

82.3 

7.9 

125.5 

60.9 

186.4 

Milwaukee  (Mitchell) 

79.1 

91.2 

7.2 

177.5 

76,2 

253.7 

( 


*/  Traffic  activity  at  secondaiy  airports  is  not  included  in  this  table 
SOURCE;  FAA  Air  Traffic  Activity  Report,  FY  1970. 
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it  is  racosjoended  that  the  criteria  for  installing  ARTS  be  amended  to 
inclode  all  aircraft  that  use  the  terminal  airspace.  Operations  at 
secondary  airports  are,  therefore,  included  in  Table  18  (p.  97).  This 
latter  table  of  operations  was  tl«  one  used  in  the  analysis  of  the  costs 
and  benefits  associated  with  the  installation  of  ARTS  III  equipment  at 
terminals  with  given  levels  of  traffic  activity. 

ARTS  III  Facility  and  Equipment  Program  Costs 

Table  3  identifies  the  procurement  and  other  facility  and  equipment 
costs  attributable  to  the  ARTS  III  program. 

Table  3 

ARTS  III  P6£  PROGRAM  COSTS 
(In  Millions  of  Dollars) 


Ihimber  of 

Lot  1 

Lot  2 

lot  3 

Total 

Facilities 

12 

23 

29 

64 

$  Contract 

$11,504 

$18,178 

$22,200^^^ 

$51,882 

$  Other 

2.696 

3.706 

1.791^^^ 

4.400 

10.802 

1.791 

$  Total 

$14,200 

$23,675 

$26,600 

$64,475 

(1)  Of  the  $26,311  million  current  contract  Lot  3  ceiling,  it  is 
estimated  that  hardware,  software  and  installation  costing  $22,200 
million  will  be  procured  by  the  agency. 

(2)  Includes  FAA  regional  engineering,  site  preparation,  spares,  factory 
inspection,  freight,  supervision  of  contractors,  on-site  activities 
and  other  "in-house"  costs. 

(3)  Lot  3  terminaticn  cost  being  held  by  Office  of  Budget. 
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DESCRIPTION  OF  THE  ARTS  III  SMOIATION  EXPERIMEKT 

A.  The  Facility 

The  function  of  the  Air  Traffic  Control  Simulation  Facility  is 
to  provide,  through  the  employment  of  simulation  techniques,  an 
environment  in  which  to  study,  research,  and  investigate  present  and 
future  ATC  systems.  These  laboratories  are  located  at  the  National 
Aviation  Facilities  Experimental  Center  (NAFEC),  Atlantic  City, 

New  Jersey.  The  recently  acquired  Digital  Simulation  Laboratory, 
through  the  use  of  its  target  generation  equipment,  data  display 
equipment  and  data  collection  equipment  provides  a  means  of  conducting 
ATC  simulations  under  laboratory  controlled  conditions  with  unprecedented 
precision.  Figure  2  is  a  photograph  of  the  displays  and  equipment  used 
in  the  experiment.  The  controller  shown  is  a  trainee  from  the  Southern 
Region. 

B.  The  Experimental  Design 

An  experiment  which  simulated  the  air  traffic  control  procedures 
at  a  simple  terminal  configuration  —  one  arrival;  one  departure  and 
one  flight  data/roordinator  position,  a  type  typical  of  the  locations 
in  the  third  lot  buy  --  was  conducted  at  the  NAFEC  facility  in  March 
1971.  The  terminal  geography  used  in  the  study  is  shown  in  Figure  3. 

Six  teams  of  certificated  controllers  were  presented  with  Identical 
traffic  samples  using  two  different  systems:  ARTS  III  and  manual.  Each 
team  had  three  members  operating  the  positions  described  above.  The 
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FIGURE  3 
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characteristics  shorn  in  Table  7.  After  a  wans-up  period  of  15  minutes 
this  table  indicates  that  the  rate  of  traffic  activity  for  each  sample 
built  to  a  total  of  36  IF2  aircraft  and  21  VFR  aircraft  duri:^  the  hour 
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SZillK  OliggrBa'S 


A  Isr^  Ecmner  of  gneasares  "sere  ostermiEed  orior  to  tiis  cosEcct  of 

2/ 

tiie  erperiaieat  to  oe  sigaificsrt  —  isssicatcrs  of  tae  cocroier  job  of 
coatrolliEg  air  traffic  ia  tiis  terasiaal  airspace.  Tcese  are  ideatified 
ia  Table  8,  aad  are  stsmnarized  beiosr  by  category:  tbe  actual  record  of 
erperiTTKrtal  results,  xaica  are  keyed  to  Table  8,  is  shawn  ia  Table  Sk. 


Ivsssx.  So.  pescriotioa  Category 


1-3 

Aircraft  Handled 

4-il 

Tire  ia  System 

12-41 

Coomuaicat  ioas 

45-48 

Advisories  (•>bserved  by  referee) 

51-56 

Conflicts 

The  above  categories  of  perforaance  were  aonitored  by  coaputer  and 
recorded  autocsatically  with  the  exception  of  category  45-48,  Advisories 
"observed  by  referee,"  It  was  felt,  prior  to  the  conduct  of  the  experi- 
cent  that  certain  critical  facets  of  safety  in  performance  could  not  be 
monitored  automatically.  For  example,  a  controller  using  ARTS  might 
have  the  additional  and  highly  relevant  information  that  a  VFR  aircraft 
with  a  Mode  C  transponder  was  at  a  conflict  altitude  with  an  IFR 


2^/  For  a  detailed  description  and  rationale  for  recording  the'^e 

measures,  see  Ref.  2.  it  is  important  to  note  that  the  specifi¬ 
cation  of  these  performance  indices  was  made  three  months  priot 
to  the  conduct  of  the  exoeriment. 


1 


Taole  8 

Description  of  Indices  of  Perfornanc^ 


Stsiber  arrivals 
jns^r  departures 
Total  arrivals  and  departures 
Coc^leted  system  time,  average  for  arrivals 
Cospleted  system  time,  average  for  departxnres 
Actual  flight  tine,  start  to  runway,  average 
for  arrivals 

Index  6  less  index  8  *  Actual  flight  time 
Actual  flight  time,  runway  to  termination, 
average  for  departinres 

Actual  flight  time,  handoff  to  runway,  average 
for  arrival 

Actual  flight  time,  runway  to  handoff,  average 
for  departures 

Total  delay  time,  average  for  arrivals 
Total  delay  time,  average  for  departures 

Time  in  communications; 

Arrival  controller  -  radio 
Arrival  controller  -  phone 
Departure  controller  -  radio 
Departure  controller  -  phone 
Controller  team  -  radio 
Controller  team  -  phone 

Requests  for  reidentification; 

Arrival  controller 
Departure  controller 
Controller  team 

Requests  for  altitude  verification; 

Arrival  controller 
Departure  controller 
Controller  team 

Directives  for  steering  vectors; 

Arrival  controller 
Departure  controller 
Controller  team 


(Cont ' d) 


Table  8  (Cont'd) 

Description  of  Indices  of  Performance 


Average  communications  time  per  aircraft: 

Arrival  controller  -  radio 
Arrival  controller  -  phone 
Departure  controller  -  radio 
Departure  controller  -  phone 
Controller  team  -  radio 
Controller  team  -  phone 

Average  requests  for  reidentification  per  aircraft: 

Arrival  controller 
Departure  controller 
Controller  team 

Average  requests  for  altitude  verification  per  aircraft 

Arrival  controller 
Departure  controller 
Controller  team 

Average  directives  for  steering  vectors  per  aircraft : 

Arrival  controller 
Departure  controller 
Controller  team 

Number  of  traffic  advisories  without  altitude 
info’T">»tion 

Number  of  traffic  advisories  with  altitude 
information 

Number  of  traffic  avoiding  actions  without 
altitude  information 

Number  of  traffic  avoiding  actions  with  altitude 
information 

Number  of  separation  standards  violations: 

VFR/IFR 

IFR/IFR 

Gross 
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Index  No. 


54 

55 

56 


61 

62 

63 

64 

65 


71 

72 

73 

74 

75 

81-89 


Table  8  (Cont'd) 

Description  of  Indices  of  Performance 

Total  time  in  violation: 

VFR/IFR 

IFR/IFR 

Gross 

Probability  a  communication  channel  is  busy 

Radio  -  Arrival  Controller 
Radio  -  Departure  Controller 
Phone  -  Arrival  Controller 
Phone  -  Dsparttire  Controller 
Phone  -  Coordinator  Controller 

Expected  duration  of  call,  given  communication 
channel  is  busy  (expected  wait  time  for  clear 
transmission) : 

Radio  -  Arrival  Controller 
Radio  -  Departure  Controller 
Phone  -  Arrival  Controller 
Phone  ”  Departure  Controller 
Phone  -  Coordinator  Controller 

Indices  of  "orderliness";  alternative  weighting  schemes 


aircraft  that  be  was  working,  A  traffic  advisory  or  a  directional 
r^tor  would,  therefore,  be  given  to  this  IFR  aircraft  —  on  a  time 
available  basis  —  in  accordance  with  the  present  legally  prescribed 
procedures  for  controllers.  On  the  other  hand,  the  controller  using 
tns  canual  system  would  be  ignorant  of  the  altitude  of  the  VFR  aircraft. 

Jib  adv^^sory  or  change  in  directional  vector  could  be  given  on  the  basis 
of  unknown  information.  A  distortion  in  test  results  would  occur.  The 
critical  avoi'^ance  of  an  unsafe  situation  would  be  recorded,  automatically, 
only  as  an  increase  in  the  time  in  terminal  for  an  IFR  aircraft.  In 
addition,  it  would  clearly  not  be  vali'  to  test  for  statistically 
si^ificant  differences  between  the  ARTS  and  manual  systems  of  control 
since  these  indices  (46  and  48)  are  logically  fixed  at  zero  for  the 
manual  system.  In  the  same  way,  no  statistical  interpretation  is  possible 
for  indices  (45  and  47)  which  measure  the  number  of  times  advisories 
were  given  unnecessarily  by  a  manual  controller  who  did  not  have  relevant 
altitude  information  concerning  VFR  aircraft  in  the  vicinity.  For  this 
reason,  it  was  decided  to  record  category  45-48,  derived  by  observation, 
in  a  separate  effort  and  to  subject  the  data  to  independent  evaluation. 

This  effort  proved  unproductive.  No  usefu?  information  was  obtained 
because,  apparently,  controllers  think  quite  far  ahead  in  maintaining 
aircraft  separation.  It  was,  therefore,  not  possible,  without  recon¬ 
structing  the  total  thought  processes  of  the  controller;  to  determine 
whether  some  specific  evasive  action  was  prompted  by  the  display  of 
altitude  information  for  a  given  aircraft.  But,  despite  the  fact  that 
catego  y  45-48  provided  lo  additional  data,  the  situations  described 
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above  were  evidently  captured  sufficiently  by  the  compu' ^r  monitoring 
of  the  other  categories  of  performances:  1)  the  evasive  actions  taken 
by  the  ARTS  controllers  were,  obviously,  contributing  factors  to  the 
observed  reduction  in  conflicts,  category  50;  2)  the  fewer  unnecessary 
advisories  given  by  ARTS  contributed  to  the  reduced  numbers  of  radio 
communications  observed  for  this  system,  category  12. 

The  following  indices  were  derived  analytically  from  the  above, 
observed,  measures: 

Ind^x  No.  Description  Category 

60-75  Communication  Channel  Congestion 

80-90  Indices  of  "orderliness" 


The  indices  of  channel  congestion  (60-75)  are  merely  translations  of 
data  previously  obtained  in  category  12-41  into  probability  estimates  that 
some  channel  of  communication  will  be  busy,  and  info  an  estimate  of  the 
expected  time  it  would  take  for  the  channel  to  clear.  This  is  a  convenient 
method  that  provides  the  analytical  platform  to  allow  for  the  exchange  of 
significant  reductions  in  Conmunications  Workload  into  the  more  critical 
currency.  Safety.  Since  it  has  been  argued  previously  that  such  a  trans¬ 
lation  involves  a  double  counting  of  benefits,  no  such  translation  of  the 
increased  probabilities  of  receiving  a  busy  signal  into  an  increasing 
probability  of  a  conflict  was  attempted.  And,  no  additional  importance 
was  attributed  to  the  fact  that  the  experiment  demonstrated  a  significant 
reduction,  at  a  high  degree  of  statistical  confidence,  in  the  probability 
that  a  channel  would  be  busy  or  to  the  fact  that  there  was  a  significant 
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reduction  in  the ‘expected  waiting  tine  for  clearance.  These  facts 
were  considered  to  be  sere  reflections  of  significant  reductions  in 
Connnunications  Workload  which  have  already  been  observed  in  category  12-41 
A  separate  and  detailed  analysis  of  category  50,  Conflicts,  was 

\  t 

undertaken  with  the  help  of  a  newly  devised  measure,  an  index  of 
"orderliness."  This  measure  was  determined  from  a  set  of  mathematical 
equations  that  attempted  to  represent  an  aircraft’s  flight  path.  By 
extending  the  mathematical  vector  of  an  aircraft's  heading  and  speed  it 
is  possible  to  predict  the  minimum  separation  distance  between  targets 
for  any  given  interval  of  time.  The  index  then  counts  all  instances  in 
w^ich  this  distance  plus  vertical  separation  are  below  established  legal 
standards.  A  violation  of  airspace  separation  standards  is  predicted  to 
occur  unless  the  aircraft  changes  its  heading,  speed,  or  altitude. 
Different  indices  were  defined  by  using  various  weighting  schemes  in 
an  attempt  to  adequately  describe  the  severity  of  the  conflict.  Are 

'  I 

conflicts  which  are  one  mile  apart  twice  as  severe  as  those  which  are 
two  miles  apart  (an  inversely  proportional  weighting  scheme) ;  are  they 

I 

four  times  as  severe  (inversely  proportional  to  the  square  of  the  dis¬ 
tance)?  Several  alternative  weighting  schemes  were  investigated.  The 
most  recent  of  these  versions  —  index  89  —  is  reported  in  Table  8. 
However,  the  conclusions  were  found  to  be  not  severely  affected  by  the 
particular  weighting  scheme  selected.  As  anticipated,  the  index  of 

2/  A  measure  defined  by  A.  G.  Halverson  of  NAFEC.  For  a  detailed 
description  of  this  measure,  see  Ref.  3. 


•’orderlisess”  wt  oaiy  prorea  to  be  £  jocd  for  as  rzrsser  of 

coofliots,  bst  to  fcs»e  csrtaia  scvzsta^s  ossr  a  direct  meescre  of  ebess 
c<Eiflicts.  For  cas  thing,  tis  saaiyticaiiy  (^rirgd  irder  of  “to^rliress” 
reflects  cb-  severity  of  the  violetioa,  rise  direct  cofrat  of  the  cnnbsr  of 
conflicts  does  not-  Indices  54-56,  Tiss  in  Conflict,  reflect  severity 
to  a  United  dsgrss.  —  i.e.,  it  is  worse  to  be  in  conflict  for  50  seconds 
ratisr  than  40  —  bnt  indices  80-90  ispart  both  a  dimension  of  tine  and 
distance  to  tlae  conflict-  Furtisr,  it  was  asstsoed  that  the  record  of 
conflicts  would  provi<^  a  psscity  of  infomaticn  suited  to  further 
analysis.  Conflicts  are  unusual  events  and  it  is  difficult  to  look  for 
patterns  and  attribute  then  to  causes  «ien  the  data  are  not  safficient 
for  this  ptnrpose.  However,  the  icdew  of  "orderliness"  is  a  hi^ly 
visible  aunber  that  can  bs  derived  analytically  at  any  instant  in  tine. 
Moreover,  as  an  index  capable  of  continuous  oeasurenent  it  is  anenable  to 
matheoatical  treataent  Ly  unsophisticated  methods.  Conflicts,  on  the 
other  hand,  are  a  discrete  variable.  At  any  instant  in  tise,  either  there 
are  conflicts  or  there  are  none.  Dependent  variables  of  zero  oagnitude 
require  special  analytical  treatment.  In  summary,  indices  80-90  are 
continuous  measures  of  large  dimension  especially  suited  for  the  purpose 
of  evaluating  patterns  of  conflict,  but  only  if  it  can  be  established 
that  they  are  an  appropriate  surrogate  for  the  actual  record. of 
contlicts. 

The  Transportation  Systems  Center  of  the  Department  of  Transportation 
was  charged  with  the  responsibility  of  determining  whether  any  of  the 
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alESro£C&I®  «»>rrr»ni»-<f  e»£t3E2:2Sg  r?-»»  «--irapy  ari  fflCT5i3rl£r&S5  caxaSi  SJt 

n.f 

cfflcsjeerexS  as  as  a^^taoriare  egraag-  zac  cagfliccs.  —  is  a  r&ssic  of 

aarf:  ic  aas  (^i^rg»-Tr."-vPf<!  rr^r  angr  g?-rj  all  ars  rri*  ala.'^rnairi'ra  scasmss 

SOC  r-g!'SrT«'8g:t-Tiri^  CbS  »fr^<?nr  (JE  '“’ip— )'^>t-'8  f irinr«'B(*  S&  3$  a  CfflCSSSiaECS 

msassre  of  coofliccs.  l&s  rao  aiaasares,  ooofliccs  arts  rns  aaoez  asf 
or(feTli2:ess,  aere  c«grr>5laora  Eo  a  estt  feigS:  sroar  of  sEarisEieal 
i^EtiET- 

«rTir.?»r  S?^  rce  eranpie  "oroarl f mss”  aeascre  iaciofac  ia  Ears  reaorE, 
is  sasaa  ia  Tsois  S*  as  rTv=!  a^srags  Talas  for  oae  near  of  siaaiaEse 
oaeraEicas.  Sines  a  <^taiisa  ezaniiaacicn  of  tfs  rscorc  of  coafl joes  aas 
ssrisicEsd  as  aa  ssssatiai  eisnsat  of  this  sEafy,  a  oiajor  aatpar  caES- 
gorj,  based  oa  Ese  of  “ors^riiasss,”  aas  added  Eo  the  list  of 

indices  sfecnra  ia  Table  S.  Aa  iastaatareoES  coaat  rf  tbe  kinds  of  targsEs 
ti^ir  aaaber,  tbs  aetbod  of  control  (i?a  or  v?S),  Ebe  type  of  traaspeneer 
vas  recorded  at  oae  aiaate  iaterrals.  These  date  tegetier  vith  tbe 
deteraiaatioa  ~f  tbe  aaaier  of  conflicts  and  a  ceriTed  estimate  of  tbe 
index  of  orderliness  •ere  recorded  for  each  controller  position.  Tbe 
conclusions  dravn  fron  an  analysis  of  this  record  »-ill  be  reported 
separately  in  Appendix  A. 

B.  Reporting  Fomat 

A  total  of  36  observations  for  each  of  the  two  controller  groups 
were  made  for  the  indices  listed  in  Table  8.  An  example  of  the  reporting 

4/  For  a  detailed  description  of  this  work,  see  Ref.  4. 
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fesif-feocr;  aaS  cj  Dstal  Sir  eSffi  asaz.  'E^ISs  9  is  &  rerenS  fee  e&s  fiSaJiS 
JaraSnesJ  groep  aS.  caarroUars,  &  gesuty  trrssz  oas  traaS  fs*  gas  larger  6$3£ 
cf  ring  errer£nerg_  l&e  miimams  in  eas  ceper  lafg  fcasd  eaggisr  cf  gee 
ezxtrj  resaeserG:  ri^  aunnsr  of  CPS  trial  brer  being  reaertsSr  brers 
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TTg^gTT  ^'^B^ISSSS 

asaraSgs 

A  <fega£Se3  aiiinmcy  as  Hrvg  <Ery£s-£:itfB?ca'B  ntssSls  Sac  sSS  Sgi3£cffis 
<5£®rrf2ffi3  fa  TT^Ie  S  £s  mwtm  celaor  as  Sa;  Sar5?z  stanfceg  fesfigas- 

cffflcffi  are  ta»  H&ls  S_  Saab  earry  iiSe  arfgSmeafc  aasaa  css 

13  ctsesTSSiccs  (6  treaiKS,  3  sanolesl  fee  eacSs  o£  era  cbo  sysrems  aS 
caccrol,  aranasl  artS  *325,  snirmeraasg  Ey  fcalf  bacr  sad  nccrlw  ccfcsls- 

3-  i^Talysis 

A  sgarasstcal  aavalysis  aegfgpsd  go  cesE  fee  sigairfeaaE  SiTf&rrerrrs-is 
ia  ere  aega  -raices  cccefasS  fee  ere  es®  sjseens  ®f  occtroi  aiss  parfeaued- 
Ifas  Isael  £rf  sSetfeeicai  iriHsf  esa  be  aesribeted  to  ary 

c6«ssi«ed  diffeeerces  ia  test  resales  is  fadicaeed  ia  'Sable  £&,  aoder 
"sigaificease.”  Ibe  mnbers  aacoa  —  .01,  .05,  etc.  —  aie  celled 
alpha  (£|  )  iraices  ead  aee  to  be  isterpreted  as  trae  rbe  cb^rred 

differences  ia  aeaa  Taloes  as  large  as  those  sbosa  ia  lable  8&.  coald  le 
eapected  to  be  dee  to  chance  oaly  (j  percent  (i.e.,  1  percent,  5  percent 
etc.)  cf  the  tire.  Ibe  standard  ”t"  test  for  detemiais^  sigaificaat 
differences  ia  aeans  vas  enplcjed.  An  analysis  of  variance  was  perfomed 
in  order  to  eliminate  those  sources  of  error  which  could  be  accounted 
for  statistically.  Tie  residual,  experinental  error,  was  used  as  tie 
denominator  for  the  "t”  test.  Those  differences  which  resurted  in 
O  values  of  0.25  or  less  are  shown  in  Table  8A.  This  aeans  that  there 
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Tinree  row  eatrees  apr^ar  for  earn  iadez  to  isaiicate 

1.  1st  half  hour  (low  traffic  activity) 

2.  2nd  half  hour  (high  traffic  activity) 

3.  Full  hour  (cozbioad  traffic  activity) 
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S&.  {Ctat*d) 


H2S533S  5SHKS3S 


CE2IIFIC^3E9 


CSBra0IF.B8S 


/i 


1  1 

|vi'TrTr*i 

SSSS  Ill 

£ieSI?iCS3SCE 

j^sis  m 

1  SI«3JIFICaSCE  j  | 

1  •  ^ 

I  i, 

1 

number  of  &ircrsrz 

1 

E  1 

10.61 

10.23 

.25 

10.61 

1G.55 

-  1 

1 

It  .50 

14.06 

13.61 

13.50 

1 

1 

25-li 

24.33 

24.22 

24.06 

1 

1  ^ 

17.44 

17.61 

17.94 

17.83 

I 

I 

19.63 

20.33 

22.50 

22.33 

1 

I 

37.23 

37.94 

40.44 

40.67 

» 

1  ^ 

23.05 

27.39 

22.56 

28.39 

1 

1 

34.33 

34.39 

35.11 

35.33 

i 

’ 

62.39 

62.28 

64.67 

64.72 

; 

tisne  is 

secoiids 

j 

A 

4 

840 

857 

.25 

812 

825 

.  4 

3 

1 

1117 

1189 

.25 

1130 

1153 

1 

J 

993 

1044 

.25 

990 

1G06 

» 

5 

1 

5 

721 

722 

703 

705 

907 

896 

827 

831 

819 

814 

771 

775 

■<! 

6 

834 

852 

.25 

805 

818 

1 

f. 

i(^ 

1172 

.25 

1095 

1127 

J 

976 

1033 

.25 

967 

991 

-25 

6-8 

216 

205 

-25 

200 

186 

317 

310 

297 

250 

.10 

274 

264 

255 

221 

.05 

7 

628 

636 

-05 

627 

627 

650 

651 

667 

665 

639 

644 

649 

649 

8 

619 

647 

.25 

606 

632 

.05 

766 

862 

.10 

798 

877 

.10 

703 

769 

.10 

712 

770 

.05 

9 

187 

197 

.25 

176 

166 

.25 

193 

196 

475 

494 

190 

196 

184 

171 

.25 

(Cont'd) 
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Table  Sk  (Cont'd) 


CERTIFICATED  OMCROLIERS 

NGN-CERTIFICATED  CONTROLIFRS 

INIEX 

H4HIIAL 

ARTS  III 

SIQUFICAKCE 

MANUAL 

ARTS  III 

SIGNIFICANCE 

22 

16.11 

3.61 

.05 

10.61 

2.56 

.05 

18.56 

5.06 

.01 

13.89 

4.44 

.05 

34.67 

8.67 

.05 

24.50 

7.00 

.01 

23 

23.22 

6.22 

-05 

13.06 

3.39 

.05 

24.61 

7.83 

.05 

16-72 

6-56 

.01 

47.83 

14.06 

.05 

29.78 

9.94 

.01 

24 

36.28 

36.39 

48.06 

46.06 

.10 

55.44 

61.06 

.25 

64.89 

71.11 

.25 

91.72 

97.44 

.25  112.94 

117.17 

25 

10.72 

8.89 

9.83 

10.00 

11.89 

12.39 

12.83 

13.50 

22.61 

21.28 

22.67 

23.50 

26 

47.00 

45.28 

57.89 

56.06 

67.33 

73.44 

.25 

77.72 

84.61 

.10 

114.33 

118.72 

135.61 

140.67 

.25 

seconds  per  aircraft 

27 

29.67 

27.28 

.10 

33.22 

31.39 

.25 

31.72 

29.78 

.25 

35.22 

35.67 

30.83 

28.61 

.10 

34.17 

33.72 

28 

11.78 

3.56 

.01 

11.11 

3.78 

.01 

10.00 

3.22 

.01 

9.00 

3.00 

.01 

10.67 

3.28 

.01 

9.89 

3.39 

.01 

29 

14.33 

12.11 

.05 

13.78 

12.56 

.25 

15.06 

12.83 

.10 

14.67 

13.56 

.05 

14.72 

12.50 

.05 

14.50 

13.28 

.05 

30 

9.44 

2.11 

.01 

10.89 

1.50 

.01 

11.72 

3.67 

.01 

11.67 

1.67 

.01 

10,56 

2.94 

.01 

11.44 

1.44 

.01 

31 

20.17 

17.67 

.01 

21.00 

19.50 

.25 

22.33 

19.61 

.10 

22.28 

21.78 

21.28 

18.72 

.05 

21.61 

20.67 

.25 

32 

10.39 

2.83 

.01 

11.17 

2.67 

.01 

12.94 

5.56 

.01 

12.94 

3.67 

.01 

11.72 

4.39 

.01 

12.17 

3.28 

.01 

(Cont 'd) 


Table  8A  (Cent  d) 


CERTIFICATED  CONTROLLERS 

NON-CERTIFICATED  COmOLLERS 

INDEX 

MANUAL 

ARTS  III 

SIGNIFICANCE 

MANUAL 

ARTS  III 

SIGNIFICANCE 

00 

.o: 

01 

.o; 

00 

.o; 

,02 

21 

01 

.01 

01 

01 

•  K 

01 

.o; 

01 

•o: 

25 

.21 

20 

11 

,00 

.01 

32 

.00 

.10 

21 

.00 

.10 

01 

.00 

.25 

01 

.00 

01 

.00 

.25 

04 

.00 

.01 

12 

.00 

.10 

09 

.00 

.05 

23 

.08 

.05 

21 

.15 

22 

.12 

.05 

59 

.14 

.05 

62 

.20 

.01 

61 

.17 

.01 

46 

.12 

.01 

46 

.18 

.05 

45 

.15 

.01 

57 

4.40 

82 

5.28 

.10 

70 

4.89 

.05 

Table  8A  (Cent  d) 


CERTIFICATED  CONTROLLERS 

NON-CERTIFICATED  CONTROLLERS 

INDEX 

MANUAL 

ARTS  III 

SIGNIFICANCE 

MANUAL 

ARTS  III 
_ 

SIGNIFICANCE 

time  in 

seconds 

64 

32 

.25 

10 

14 

85 

58 

.05 

65 

87 

149 

90 

.05 

75 

101 

37 

19 

.25 

65 

40 

178 

182 

354 

142 

214 

201 

419 

182 

238 

171 

.05 

262 

220 

767 

736 

947 

781 

1005  907 

probability 

1209 

1001 

.176 

.157 

.01 

.193 

.184 

.253 

.231 

.25 

.262 

.267 

.215 

.193 

.05 

.227 

.225 

.141 

.118 

.05 

.137 

.124 

.168 

.146 

.25 

.183 

.172 

.152 

.132 

.10 

.159 

.149 

.068 

.020 

.01 

-066 

.022 

.078 

.022 

.01 

.067 

.021 

,074 

.022 

.01 

.066 

.022 

.091 

.021 

.01 

.108 

.015 

.126 

.042 

.01 

.147 

.019 

.108 

.031 

.01 

.127 

.018 

.002 

.003 

.006 

.004 

.037 

.043 

.046 

.031 

.021 

.023 

.025 

.018 

(Cent 'd) 

Table  8A  (Cont'd) 


1  CSSriFICATED  COKTROLLERS 

NON-CERTIFICATED  CONTROLLERS 

1  iiHEX 

3«S3JAL 

ARTS  III 

SIGNIFICANCE 

MANUAL 

ARTS  III 

SIGNIFICANCE 

tire  in 

seconds 

71 

2.14 

2.04 

.25 

2.40 

2.42 

2.19 

2.01 

.10 

2.33 

2.42 

.25 

2.18 

2.03 

.10 

2.36 

2.42 

72 

2.47 

2.37 

2.69 

2.70 

2.21 

2.20 

2.50 

2.59 

2.33 

2.28 

2.58 

2.64 

73 

7.63 

5.65 

.01 

7.25 

6.33 

.05 

9.71 

6.59 

.01 

8.24 

6.32 

.01 

8.81 

6.25 

.01 

7.80 

6.52 

.01 

74 

10.14 

7.15 

7.91 

7.25 

8.66 

8.38 

9.47 

7.80 

.10 

9.69 

8.31 

8.82 

8.67 

75 

1.60 

3.00 

3.69 

1.96 

.25 

11.66 

10.91 

7.76 

5.32 

.10 

11.72 

10.88 

7.76 

5.37 

.10 

index  of  orderliness 
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is  a  statistical  likelihood  of  three  out  of  four  that  the  observed 
differences  were  due  to  some  functional  cause;  not  chance. 


A  short  note  is,  perhaps,  in  order  to  remind  the  reader  that 
the  Cl  level  at  which  an  observed  difference  is  considered  to  be 
significant  is  not  a  matter  to  be  settled  by  the  statistician. 

A  rational  decision  requires  that  the  practical,  not  statistical, 
significance  of  a  difference  be  considered.  Only  the  manager 
responsible  for  choosing  between  alternative  systems  of  equipment 
can  assign  the  appropriate  practical  value  to  be  placed  on 
differences  between  candidate  systems.  To  repeat,  the  statistician 
can  make  quantitative  estimates  of  the  differences  in  means  and  he 
can  indicate  the  likelihood  that  these  differences  are  attributable 
to  real  or  functional  causes,  if  certain  assumptions  about  the 
frequency  distribution  of  events  are  made.  But,  the  relevant 
determinant  of  differences  between  systems  must  always  depend  upon 
the  costs  involved  in  making  either  of  the  two  possible  types  of 
statistical  error;  rejecting  a  hypothesis  that  the  systems  are 
not  different  when,  in  fact,  they  are  different,  b)  accepting 
a  hypothesis  that  the  systems  are  different  when,  in  fact,  they 
are  not.  The  costs  of  making  a  mistake  in  either  of  the  above 
two  ways  determine  the  practical  significance  of  the  experimental 
results.  The  statistician's  tools  can  tell  us  nothing  about  this. 


The  costs  involved  in  accepting  the  hypothesis  that  ARTS  equipment 
results  in  fewer  conflicts  than  the  use  of  the  present  equipment  when, 
in  fact,  it  does  not,  are  the  costs  associated  with  buying  more  expensive 
equipment,  needlessly.  However,  if  we  reject  the  hypothesis  that  ARTS 
is  safer  when,  indeed,  it  is,  then  we  are  making  a  mistake  of  greater 
dimension.  In  order  to  guard  against  this  very  costly  error  —  the 
precise  degree  of  cost  depends,  remember,  on  the  subjective  judgment  of 
the  person  charged  with  making  the  decision  --  we  should  be  willing 
to  accept  the  premise  that  ARTS  is  better  than  manual  at  a  practical 


level  of  significance  that  is  in  considerable  exces§  of  statistical 
significance. 

On  the  other  hand,  there  are  cases  in  which  the  practical  level  of 
significance  may  be  less  than  some  conventional  level  of  statistical 
significance.  For  example,  indications  of  differences  in  comtmicat ion 
workloadSj  especially  those  involved  with  talephoiie  conamnications  have 

less  practical  significance.  Ignoring,  for  the  momen*',  any  ancillary 

1 

considerations  of  safety  dhe  to  the  fact  that  lines  may  be  busy  at 
inopportune  times,  differences  in  the  number  of  telephone  conmunicatior.s 
can  alwa  s  be  accommodated,  cheaply,  by  installing  more  libes.  (Not  so, 

i 

perhaps,  for  radio  conmunicat ions ,  especially  if  the  limit  of  available 
frequencies  has  been  reached.  However,  this  limit  is  not  likely  tp  be 
reached  at  any  time  in  the  near  future  for  the  specific  terminal 
locations  in  our  study.)  For  this  reason,  we  should  insist  on  very 
high  levels  I  of  statistical  significance  before  we  accept  the  premise 

I 

that  ARTS  requires  less  time  in  phone  communication  than  the  manual 
system.  The  level  of  statistical  significance  determined  by  experiment 
did  prove  to  be  very  high;  Q  =  0.01. 

2/  A  succinct  description  of  the  point  we  are  making  can  be  found  in  the 
New  York  Daily  News  of  May  15,  1972,  (p.  20).  Under  the  headline 
"Heart  Drug  Dropped  After  Death  Rate  Study"  there  appears  the 
following  quotation  attributed  to  Dr.  Jeremiah  Stamler,  a  noted 
cardiologist  at  Northwestern  University  Medical  School  in  Chicago: 
"The  number  of  deaths,"  he  declared,  "was  not  statistically 
significant  but  one  prefers  not  to  wait  for  statistical  signifi¬ 
cance  --  you  quit.  In  medical  research,  above  all  there  is  a 
responsibility  to  avoid  harm.  That  is  why  we  made  the  decision." 
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Host  isporteatly,  120  data  revealed  tis  experiaeat  radicate  tbat 
ASTS  is  less  safe  than  the  eantsal  ^-scea.  There  is  so  evideace,  i2t!0!ere.r 
inplicit,  that  ARTS  dees  ham. 

To  the  contrary,  the  evidence  indicates  that  tbs  potential 
for  increased  safety  is  probably  being  understated  in  tbs  table 
of  results  presented.  If  inferences  are  dravn  frea  tie  entire 
second  half  of  the  expexinent  which  used  trainee  controllers, 
the  potential  exists  for  the  certificated  controller  to  do  even 
better  in  their  ability  to  separate  i?R  x  IFR  aircraft-  It  can 
be  argued  that  the  trainee  controller's  improved  performance  with 
ARTS  was  the  result  of  this  group  being  less  acenstoaed  to  the 
manual  system.  The  opportunity  to  discard  old  habits  and  to 
utilize  the  automated  system  core  fully  will  be  realized,  in  tire, 
to  a  greater  extent  by  the  certificated  controller. 

It  is  also  essential  to  point  out  that  it  was  never  the  study's 

intention  to  analyze  the  experirental  data  for  isolated  instances  of 

statistical  significance.  Scxre  240  measures  were  recorded  —  about 

80  separate  indices  in  three  tioe  periods  —  and  it  would  be  a  sinple, 

as  well  as  incorrect  matter  to  highli^t  those  indices  which  favored 

ARTS  to  a  statistically  significant  degree,  and  ignore  all  others.  We 

have  already  indicated  how  statistical  significance  can  be  an  irrelevant 

criterion  for  decision  making,  but  since  this  was  the  criterion 

established  in  discussions  with  the  panel  of  reviewers  in  the  Department 

of  Transportation  and  the  Office  of  Management  and  Budget,  we  had  no 

choice  other  than  to  accept  their  dictate  of  t^is  criterion.  However, 

we  voluntarily  imposed  an  additional,  more  stringent  and  valid,  standard 

of  analysis  based  upon  logical,  and  not  merely  statistical,  considerations. 

We  attempted  to  determine  the  causal  or  inherent  differences  between 

systems.  Explanations  or  hypotheses  to  explain  these  anticipated 
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differences  sere  procsssed  Sefcare  «rrtg  ssS.  l^^l»  eagerfnerxr  -  Baffereaces 

o&serred  ss  cerftmiarace  meesrres  £sad  co  ce  acinnt=2ed  fer  s.  cessel  iigic 
to  s  real  difference  oetseea  asSS  memsl  sgrstems-  For  eranple, 
it  was  anticipated  tzaat  tfee  sn&stantire  casstricctioa  ms£fs  by  III 

e^ipmeat  was  the  casitimsal  displ^  of  infuCTatioa  relevant  to  rhe  cErtrol 
of  aircraft:  alplsnrnieric  l&nEiey,  grcwtd  speed  and  alEitcde,  It  was 
eapected  that  a  ccntroiler  soald  react  to  this  iafomatioa  hy  fr:g3rii7^  aa 
increased  amrSer  of  actirais  to  atoid  conflicts;  actions  which,  ocnriocsly, 
would  not  be  tasea  in  the  absence  of  sneh  iarornatica.  Suffice  it  to  ssx, 
we  did  observe  that  index  26  —  "directives  for  steerirg  vectors”  —  were 
higher  for  the  *315  systen.  Sitb  EDore  steering  vectors  given  uader  ASTS, 
ccasisteat  with  ig^potbesls  that  t\&  data  block  is  relevant  to  the  control 
of  aircraft,  there  is  then  no  logical  explanation  to  account  for  an 
expectation  that  the  tire  spent  in  terainal  control  —  index  8  —  would 
be  less  using  ARTS.  The  experisent  verifies  that  it  is  not. 

Finally,  we  inposed  a  standard  for  the  analysis  that  is  sinilar  to 
the  standard  of  statistical  significance  in  that  it  was  dictated  to  us 
by  the  panel  of  reviewers,  and  is  equally  susceptible  to  spurious  or 
irrelevant  conclusions.  This  additional  standard  linited  the  analysis 
to  only  those  differences  which  could  be  translated  into  dollar  benefits. 

Throughout  history,  the  essence  of  the  contribution  made  by 
automation  from  the  days  before  the  invention  of  the  plow,  to  the 
sewing  machines  of  the  19th  century,  and  including  the  wonders  of 
today's  technology  has  been  the  ability  to  reduce  the  variation  in 
individual  performance.  We  now  live  in  a  world  where  few  of  us 
ever  know  the  pleasure  of  nibbling  at  a  tub  of  great  tasting  butter, 
but  we  seldom  are  also  disappointed  in  the  other  direction.  "Homoge¬ 
nized"  is  the  label  that  typifies  today's  products  and  services.  Our 
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$$es2  (Srigargg  ansKsSoes,  mnst  gcofe^ly,  sse  sgill  ccKt  ^2$  tro 
campege  witn  g£ffi  2ese  ard  S^saiary  3s£ra  ^rry,  fectt  I±j2y  caa  ©car 
cg£a'<e  eaierjctas  else,  ic  as  eos  aaalfsy  co  cso&ass  aa  caiasySrs 
siaaiard  erf  gearfosaesatg;  from  s.  larger  nanriiigr  esf  senSbers  tfeag  £$ 
t±e  cro2  aaSracle  of  aatomagiaa.  So,  coo,  araira  SSSS  HI,  ic  was 
eamecced  seme  apSicr&eaal  eaca:  nailer  csirrg  older  nreefamis  ooaM 
egrperfozim  ceber  cacr nailers  csScg  aaremated  ce^aicaes-  "Efe  aiore 
fmporcaac  qaesclon  oas  atfecner  all  canrcrollers,  faElnSirg  ebe 
loiFliesE  perforner,  canid  £e  fenac^a  co  e  standard  gbgr  (fenKrescraced 
a  aarrocT  targe  in  perfoansree  for  all  c^rs  of  cbe  ecnipmeac.  More 
efficient  olsnai^g  for  fg»t* mamcBter,  ig>«  selectire 
reemicirg  tneefoads  fronn  elite  papolacion  grocas  only,  waoM  resale - 
llaerefore,  pifijr'*isioa  tag«:  mgds  £©  ssaljcze  scandard  dsriation  — 
a  scacistical  measnre  of  Ces  -mariation  in  indiwidcai  cotsCroiler 
performanre  —  for  every  index  specified  in  'Esble  S-  BrC,  fespite 
the  iaxporcarce  of  cMs  measnee,  ciiere  is  no  objeccire  way  Co  assign 
doiiar  berefics  co  iC-  For  exanple,  ic  is  possible  to  resort  that 
tee  standard  fewiatioa  for  index  54  —  "tiaae  in  conflicts  for 
iF3.  a  ^F3  aircraft”  —  aias  SO  seconds  for  fno  certificated  con¬ 
troller  csirg  A2IS  and  127  seconds  for  the  manral  controller. 

S?sed  epon  tne  above  cisenssion  of  snat  it  is  that  antemation 
really  ccatribntes,  it  laonie  seem  that  this  is  nsefal  and  relevant 
information  to  tne  decision  maker,  bnt  since  it  camot  be  expressed 
as  a  dollar  benefit,  no  listing  of  ^viatioss  recorded  for  tee 
stndy's  indices  of  performaiSEe  is  provided. 

For  tbe  saze  reason,  there  was  no  way  to  assign  dollsr  benefits  to 
ARTS*  ability  to  readily  identify  emergency  situations.  Special  codes 


assigned  to  a  variety  of  flight  emergencies  are  an  invaluable  aid  to 
air  traffic  control.  However,  it  was  ziot  possible  to  simulate  emergency 
sitizations  in  the  present  study-  And,  even  if  we  could  simulate  these 
situations  tbe  result  would  be  a  perverse  one  which  found  that  ARTS 


benefits  are  negative.  tL_  only  way  that  an  emergency  situation  could 
be  reflected  in  our  study  would  be  in  the  increased  communications 
v’orkload  recorded  by  the  ARTS  facility  working  the  energency. 


6/  The  conplete  body  of  data  for  all  indices  is  available  from  the  FAA 
Office  of  Aviation  Economics  to  anyone  with  a  legitimate  need  to 
know.  The  calculation  of  the  appropriate  measures  of  standard 
deviations  can  be  made  from  this  body  of  data.  This  calculation  and 
their  interpretation  are  left  as  exercises  for  the  interested  reader. 
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■gSe  iraane  osielcsgsd  ia  eacs  ssafj,  toe  rig>s<j  g©  eJiaarsse  riSa  nrc&Isn! 
of  •’feanDOTra'*’  tar^lts,  is  a  profcc=d  ace  sisat  eaferaaes  tnacj  alteEcscice 
soic2:iaE^.  IC  is  Tv=ir)ag:<ar-g  stress  rfta  siansiateis  ercensierc 

soica  aas  coscacted  at  sSfEC  tsas  iimrited  to  a  <faaice  oetseea  tso  levels 
of  actooiatioa  day^-r»sd  as  52ZS  iif  ar-ir9  ataarai.  3ih>  otier  altesBaticss 
sese  cossideted.  As  a  resalt,  tisis  sto%  is  sileat  corceraicg 
effect  icetess  of  asj  os^rer  aitersat£^?2s.  I£e  problems  aad  tice  isrorred 
ia  iisstitatirs  chss^es  ia  termiaal  procedures  or  seBaratica  stasdards 
are,  iikesra.se,  cckccsfa  aad  not  studied.  Hi?ss»er,  tiie  erpericeat 
demoastrated  that  conflicts  resultii^  frca  tee  abo*e  interaction  could 
be  reduced  vitfa  no  changes  sraatevar  in  current  procedusres  and  practices 
of  teminal  control.  Moreover,  changes  in  procedure,  considerations 
of  teminal  control  areas  or  ether  cethods  for  separating  aircraft,  are 
not  really  alternatives  to  the  introduction  of  ARTS  III.  An  ironic 
conclusion  of  '•he  study  is  that  instead  of  it  being  sore  difficult  to 
justify  the  installation  of  ARTS  III  at  the  less  dense  temiaals,  the 
installation  of  a  technological  (automation)  solution  to  the  problem 
of  IFR  X  VFR  interactions  makes  more  sense  at  these  terminals,  ARTS  III 


Ss,  gns£iz5>aj,  trctr  a  ssffScSesz  golarfco  s:  gae  arose  (Sesate  ggoarggclis- 


•feSs  dsesE  ffott  tneaa,  oatffisseriSy-  cSisr  15?  «3e-  £S3S  H  wccsSd  be 
bffiaaiffisr-  Ic  meaers  g&gr  s  pzatsssSsrsl  dhs:^  £s,  ggoeagoSy,  ose^r-j  c® 
cemplanesg  ggffi  farscoseoescrs  £n  esoinnasg.  Ibe  ssrcssgian  goilgir  £n  ere 
^ilicy  e®  <££^1^  trsz^szs  —  ere  Sluigs  fa  beasoo  ttecbonifogy,  cgmcctrer 
czofficfEy,  ggeraeser  arrfcfrg  aoi3  ere  cotrertsllsr's  Galley  co  asramSSeee 
rnta  ^<Tr^  foT  frfTps  ^i-t*  ccanners  ffl£  sre  riiig 

"^coag”  e®  Ssfaji  —  ■arili,  aaaacaoeSlj,  be  a  arajerr  pra&Isn!  recafrfrg 
asdiefocal  aieasoresE®  sofee  ccaofen!  cf  erfeoca  afrerafe  ae  majtsr 
cermfrals.  £SSS  Ui  ariii  allea  for  aa  fnrezriac  soiegfoOa  a  creatcirg 
space,  siiie  ebes?.  aieerraerses  are  scegb^.  --»£  ere  fcrearMrg  space 
asiii  be  grearesE  at  ebe  less  (derse  teramrals.  "ZEisrefore,  a  major 
fmplicatioQ  of  tois  sEcny  is  Ebat  i£  is  recessarjF  Eo  bare  a  coocerEed 
canEoitmerE  Eo  Ebs  solaEioa  of  Ebe  problem!  of  EB  x  ¥5S  iateracEjosss . 

ASTS  ill  is  Irdi  cared  as  a  feasible  solcEioa  to  this  prcblemi  at  tbsose 
locations  located  in  the  third  lot  boj. 

C-  Hi^limts  of  the  Test  gesclts 

The  salient  features  :»f  tee  ezperireat  are  surnarized  in  Table  10. 

In  the  next  section,  D,  a  detailed  discussion  of  those  ezperiaental 
results  that  are  cost  critical  to  the  study's  conclusions  will  be 
presented.  This  section  summarizes  the  highlights  shown  in  Table  10. 

I.  The  operations  coiiat  —  the  total  number  of  arrivals  and 
departinres,  index  3  —  handled  by  controllers  did  nor  differ  significantly 
between  the  ARTS  and  manual  systems. 
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■Sb*s  gaartr  did  nmtt  dSfiSer  as  well  eegweea  l±e  trwo  csacanrasller 
Sswqps:  egrgSfgisesed  a=d  crafree.  iSe  date  fisr  e&g^BS  srgegs  were 
albscgs  aaaJjzaS  ga?ffirataly-  3S3>  paolfgg  tsS  resales  tcera&  plate 
bgcsffga  gggeps  £a  order  co  increese  tme  lifcelibnod  of  demBcsereeirs 
sestiscical  signSficante-  Btwewer,  wtere  appropriare,  iriingTg-rrfg^ 
were  msde  feesweea  groeps,  wisnaca:  eecrin-jrirg  sgaefseical  si^eifi- 
cante,  in  order  eo  sdSidonel  ftfooiecifflni  n®csr  iriHg 

gamceEirg  sysrens  naccrol  craffic. 

II-  'Bite  cine  in  srsEem  for  atrrgfrg  aircrafe,  irdez  6  —  measosed 
fronn  rite  rsme  c£e  aircrafe  enEer  ene  experinenE  from  a  ‘'^^aasE”  posicion, 
represenEieg  an  ea  rotrEe  cocEroi  cecEer  adJatenE  to  Er-a  tenisinal,  mEil 
c£e  final  appreagfo  co  tie  r^aga^-  —  was  M^ssCa  to  a  margi rally 
significant  (fegree,  (  Q  =  0,23)  for  iSSS  III. 

Ite  tine  in  tie  terminal  system  for  arriwiag  aircraft,  igdeT  S  — 
jneasnred  frcno  tie  tire  of  Eire  isndoff  Eo  tie  Eermisal  controller  cntil 
tie  final  g??roaci  to  tie  mnsay  —  was  ii^iar  to  a  utore  significant 
degree  (  a  =  0.10)  for  asiS  lil- 

■•ie  abore  data  for  tie  tice  ia  system  ccafirm  that  results  were 
causally  linked  to  differences  in  ecuipwent.  A2TS  equiprent  hss  such 
features  as  an  autccatic  handoff  of  control  between  the  center  and  the 
texninai  and  an  automatic  handoff  between  sectors  of  a  temlnal.  Therefore, 
based  on  the  results  shown  for  the  tire  in  system  there  is  strong  reason  to 
believe  that  the  observed  differences  between  systems  were  not  haphazard 
events  but  ratter  a  confirmation  of  the  fact  that  a  valid  experiment  took 
place.  For  example,  the  separate  indices  measuring  the  time  in  system 
(numbers  6  and  8)  both  indicate  that  aircraft  controlled  by  ARTS  were  in 
the  system  longer.  But,  the  difference  In  these  indices  (6  minus  8) 
measures  the  time  it  takes  to  transfer  control  from  the  en  route  center 
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to  tbs  terssissl;  and  tbis  rnesstse  favors  .&3ZS.  IsSsz  (6  saTtss  8)  is 
cfijser^fsd  to  be  less  for  AB3S  becajise  this  ^stea  cses  eqcipireat  desig:^ 
to  provide  aa  aiitcaatic  baadoff  capability.  For  this  sare  reason,  it  is 
argced  that  the  indices  cseasoricg  tbe  tire  spent  in  corzamicatioa 
(12  tbrongb  17)  are  loser  to  a  statistically  significant  degree  for  tbe 
ASIS  systen  because  of  physical  differences  in  the  equipnent  used. 

This  is  especially  true  of  those  indices  neasuring  phone  co-ramications 
(inder  17;  fl  =  .01),  Again,  ARTS  ecn)loys  aa  autccatic  handoff  feature 
betaeen  teminal  sectors,  and  less  tine  is  needed  for  controllers  to 
coordinate  these  handoffs. 

Indices  27-30  neasure  ccrmtnicat ions  on  a  per  aircraft  basis,  but  the 
mcAer  of  operations  handled  did  not  differ  lauch  between  systens  so  that 
these  differences,  while  statistically  significant  in  favor  of  ARTS, 
cserely  confira  the  pattern  previously  reported  for  indices  12  through  17. 

Finally,  in  demonstrating  that  results  are  not  likely  to  be  random 
occurrences  which  are  not  related  to  inherent  differences  in  the  systems, 
the  number  of  requests  for  altitude  verification  (index  23)  and  target 
reidentification  (index  20)  were  significantly  less  for  the  ARTS  system. 
ARTS  equipment  was  designed  to  provide  these  reductions  by  its  continued 
display  of  altitude  information.  The  experime  '■  ''.onfirmed  our 
expectations. 


III.  The  experiment  revealed  that  the  use  of  ARTS  III  equipment  is 
likely  to  result  in  a  reduction  in  the  number  of  violations  of  airspace 
(conflicts) . 


a)  Icdez  52,  the  nudber  of  conflicts  involving  IFR  x  IFR 
aircraft  were  observed  to  be  reduced  for  the  certificated  controller 
using  ASTS-  lliis  reduction  is  indicated  to  be  of  great  practical 
significance  although  not  statistically  significant  at  an  £7  level  of 
0.25.  A  detailed  discussion  as  well  as  a  sasiple  by  sample  cceiparison 
of  results  for  this  index  will  he  provided  bexow  in  section  D. 

This  same  index  was  significantly  lower,  in  a  statistical  sense, 
for  the  non-certificated  group  of  controllers  ( q  =  0.05  for  the  com¬ 
bined  hour) .  However,  in  keeping  with  the  ground  rules  established  for 
the  study,  no  cross-pooling  of  results  between  groups  took  place.  The 
finding  of  a  statistically  significant  reduction  in  index  52,  a  sensitive 
measure  of  performance, was  used  only  to  infer  in  a  qualitative  way  that 
certificated  controllers  could  be  eiqiected  to  do  better  with  ARTS;  to 
improve  by  an  amount  equal  to  that  shown  for  the  trainee  group  after 
they  had  thrown  off  the  inertia  brought  about  by  the  habitual  use  of  the 
older  system.  But,  in  addition,  this  finding  indicated  the  possibility 
for  a  modest  dollar  benefit  attributable  to  ARTS  because  of  the  demon¬ 
strated  ability  to  reduce  the  time  required  to  train  controllers. 

b)  Index  54,  "The  Time  in  Conflict  for  IFR  x  VFR  Aircraft,"  was 
observed  to  be  significantly  reduced  for  the  certificated  controller  to 
a  high  degree  of  statistical  significance  (  0  =  0.05)  for  the  half-hour 
of  high  activity  as  well  as  for  the  combined  hour  of  traffic.  Time  in 
Conflict  incorporates  a  measure  of  the  severity  of  a  violation  in 
separation  standards,  and  this  finding  was  considered  to  be  the  most 
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important  of  the  entire  set  of  experimental  results.  The  obseirved 
difference  in  this  index  meets  all  of  the  evaluation  criteria  we  imposed. 
The  difference;  1)  was  observed  to  talce  place  while  certificated  con¬ 
trollers  were  operating  the  equipment,  2)  was  determined  to  be  statis¬ 
tically  significant,  3)  is  logically  tenable  and  causally  linked  to 
differences  in  the  equipment,  and  4)  is  translatable  into  dollar 
benefits.  This  translation  of  an  experimental  finding  into  an  estimate 
of  dollar  benefits  is  shown  in  section  III  below. 

•  c)  Index  55,  The  Time  in  Conflict  for  IFR  x  IFR  Aircraft, 
indicates  that  the  trainee  group  performed  poorly  with  the  manual  system. 
Their  rate  of  time  in  conflicts  for  the  high  half-hour  of  activity  was 
an  average  of  708  seconds,  (2  x  354),  or  11.8  minutes  out  of  each  hour 
of  active  traffic.  The  finding  that  trainee  controllers  were  in  con¬ 
flict  approximately  20  percent  of  the  time  while  using  the  manual  system 
was  taken  to  be  another  indication  that  reductions  in  the  time  to  train 
controllers  were  possible  with  ARTS. 

d)  The  comparison  of  the  indices  of  orderliness  was  found  to 
favor  the  ARTS  system.  This  index  was  proposed  in  order  to  correct  for 
anticipated  deficiencies  in  conflict  data.  Midair  collisions  are  rare 
events,  and  conflicts  in  airspace  are  not  likely  to  occur  with  great 
frequency.  After  all,  the  entire  system  of  control  --  men,  equipment 
and  procedures  —  is  designed  to  prevent  conflicts.  This  is  desirable 
from  everyone's  point  of  view  except  the  dispassionate  analyst  who  cannot 
use  his  tools  of  analysis  to  discern  patterns  and  differences  in  conflicts 
when  the  data  are  few.  Previous  experience  with  a  simulation  of  the 


Baltimore  terminal  area  — ^  did  not  yi6ld  conflict  data  of  sufficient 
accuracy  and  quantity  to  permit  an  objective  judgment  of  results.  For 
this  reason,  it  was  felt  that  a  proxy  measure  should  be  used.  This 
measure  was  designed  to  count  all  the  Instances  in  which  there  would  be 
a  conflict  unless  a  controller  intervened.  This  index  predicted  the 
future  path  of  all  targets  under  a  controller's  jurisdiction,  and 
recorded  all  instances  in  which  minimum  standards  of  lateral  and  vertical 
airspace  were  violated.  These  instances  were  aggregated  into  a  composite 
index  with  the  use  of  a  somewhat  arbitrary  weighting  scheme  to  reflect 
the  severity  of  the  violation.  It  was  felt  that  this  would  be  analogous 
to  measuring  the  comparative  number  of  commands  being  given  by,  say,  a 
traffic  policemen  at  a  busy  Intersection.  It  would  be  i.rultless  to 
compare  the  relative  numbers  of  accidents  or  collisions  ut  such  inter¬ 
sections.  But,  it  would  be  a  meaningful  comparioon  to  record  the  numbers 
of  commands  that  had  to  be  given  in  order  to  provide  for  a  safe  inter¬ 
section.  In  the  same  way,  it  was  anticipated  that  actual  conflicts  in 
the  terminal  airspace  would  be  rare,  but  the  study  intended  to  argue  that 
a  system  which  required  fewer  commands  on  the  part  of  the  controller  in 
order  to  avoid  conflicts  would  be  a  more  "orderly"  and,  hence,  a  safer 
system  than  one  which  required  a  greater  number  of  commands. 

Index  89  is  a  composite  index  of  orderliness  for  both  the  arrival 
and  departure  positions.  It  shows  a  statistically  signifioant  reduction 
in  favor  of  ARTS  (flf  “  0.25). 
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Alehgagh  't  was  anticipated  that  the  demonstration  of  a  favorable 
restdt  in  this  index  would  provide  a  strong  argument  for  AS.TS,  when  the 
eaa-ire  pac5:age  of  results  was  analyzed,  it  was  discovered  that  this  index 
was,  issued,  an  important  ensure,  hue  for  a  different  reason.  For  one 
thing,  there  was  no  lack  of  conflict  data  generated  by  the  experiment. 
Faiike  previous  attempts  at  simulation,  the  equipment  used  in  the 
orsseat  experiment  was  vastly  superior  in  that,  for  the  first  time, 
traffic  could  be  presented  in  digitized  form  with  sufficient  accuracy  to 
enable  conflicts  to  be  measured  directly.  Because  conflict  data  did 
sps&sr  in  ample  quantity  to  allow  for  statistical  judgments  to  be  made, 
there  was  now  no  rTTgent  need  to  resort  to  a  demonstration  of  differences 
in  a  substitute  measure,  the  index  of  orderliness.  But  it  was  possible  to 
reap  other  analytical  benefits  by  attempting  to  establish  the  validity 
of  this  ceasure.  A  "spin-off”  anv~^ysis  was  performed  to  relate  this 
index  to  the  conflict  data  in  order  to  determine  whether  it  measured 
sons  important  facet  of  air  traffic  control,  or  whether,  perhaps,  it  was 
merely  an  arbitrary  measure  that  fluctuated  at  random? 

Two  separate  pieces  of  evidence  indicate  that  it  is  more  reasonable 
to  conclude  that  the  index  of  orderliness  is  related  to  essential 
functions  of  air  traffic  control: 

8  / 

1)  The  investigations  by  the  Transportation  Systems  Center  — ' 
demonstrate  a  very  high  correlation  between  this  index  and  the  number 
of  conflicts. 


8/  op.  cit.  Ref.  4. 
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2)  At  the  end  of  the  experiment,  A.  G.  Halverson  ranked  the 
performance  of  all  controller  teams  engaged  in  the  simulated  experi¬ 
ment  solely  on  the  basis  of  his  index  of  orderliness.  An  indep*-ndent 
ranking  was  also  made  of  controller  performance  by  D.  0.  Brown,  the 
project  manager  for  the  NAFEC  experiment.  The  rankings  were  almost 
identical. 

With  the  evidence  of  statistical  equivalence  as  well  as  indications 
of  functional  validity,  it  was  decided  to  use  the  index  of  orderliness 
as  a  proxy  for  conflicts  in  subsequent  analytical  investigations  of 
the  probable  causes  for  conflicts  in  terminal  airspace.  An  Instantaneous 
count  of  all  targets  on  the  radar  scope  was  made  at  one  minute  Intervals 
for  the  dinration  of  the  experiment.  Since,  in  any  given  minute  it  is 
possible  that  no  conflicts  would  occur,  the  analysis  of  these  conflicts 
would  require  sophisticated  mathematical  manipulations.  Statistical 
regression  analyses  which  relate  traffic  characteristics  to  a  dependent 
variable.  Conflicts,  for  which  zero  values  are  possible,  can  be  quite 
messy.  Not  so,  when  the  index  of  orderliness  is  substituted  as  the 
variable.  A  detailed  discussion  of  the  subsequent  investigations  of 
the  causes  of  conflicts,  using  the  index  of  orderliness  as  a  substitute 
measure  of  these  conflicts  is  presented  in  Appendix  A. 

IV.  The  investigations  into  controller  capacity  indicated  no 
statistically  significant  differences  between  systems.  Alternative  ways 
of  defining  capacity  and  estimating  its  dimension  were  tried.  A 
detailed  description  of  these  efforts  is  provided  in  section  D,  below. 


! 


In  general,  attempts  were  made  to  estimate  the  comparative  time  it  took 

i 

each  system  to  become  overloaded  and  to  go  into  "ho7.ding.'' 

V. •  Analytical  extrapolations  of-  the  experimental  data  were  com¬ 
pleted  successfully  and  are  reported  in  detail  in  Appendix  A.  In 

II 

summary,  these  analytical  investigations  were  performed  to  relate  the 
observed  number  of  conflicts  to  characteristics  of  the  traffic  sample: 

f 

1)  the  density  of  traffic  by  type:  IFRi  VFR;  2)  the  proportion  of' 
targets  having  transponders  with  Mode  C  capability.  These  investigations 

.  »  .  I 

•  indicate  that  the  number  of  unknown  targets  were  a  statistically 
significant  variable  contributing  to'  conflicts  and  more  importantly,  . 

I  I 

that  there  was  a  statistically  significant  reduction  due  to  ARTS  in  the 
number  of  conflicts  associated  with  increasing  levels  of  unknown  traffic. 
In  addition,  these  investigations  indicate  that  the  so-called  "gas 
molecule"  law  which  theorizes  that  conflicts  will  increase  as  the  square 

\  I  ^ 

of  the  number  of  airborne  targets  does  not  apply  to  the  case  in  which 
these  targets  are  under  the  discipline  of  an  air  traffic  control  system. 

,  '  i 

Conflicts  were  observed  to  increase  at, a  constant  percentage  of  unity 
as  IFR  traffic  increased;  an  exponent  closer  |to  1.0;  not  2.0.  For  these 
investigations,  the  number  of  conflicts  were  represented  by  the  index 

I 

of  orderliness.  However,  the  reliability  of  the  results  of  this  follow- 
on  analytical  effort  is  questionable,  and  will  be  discussed'  as  a 
separate  topic  in  Appendix  A.  ' 

( 

D.  Detailed  Analysis  of  the  Test  Results' 

} 

In  th:^s  sections  a  detailed  discussion  of  results  will  be  presented 
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of  the  folloving  topics:  a)  Controller  Capacity  and  b)  Conflicts: 

A  Measure  of  Safety. 

a)  Controller  Capacity 

If  one  could  meastnre  the  rate  at  which  inputs  were  coming  into 
a  system,  and  the  rate  at  which  they  were  leaving,  it  would  he  a  simple 
matter  to  define  the  capacity  of  the  a  as  the  point  at  which  the 
arrival  rate  exceeded  the  exit  rate,  in  effect,  this  definition 
measures  when  a  system  "backs  up,"  or  in  the  case  of  terminal  control 
when  it  goes  into  "holding;"  "R,"  as  defined  below  exceeds  1.  But, 
these  attempts  performed  by  The  MITRE  Corporation  were  not  successful. 
It  seems  that  from  the  very  first  minute  of  the  simulation,  the  rate  of 
arriving  aircraft  exceeded  the  rate  at  which  aircraft  were  exiting  the 
system. 

a  rate  of  entry  of  aircraft 
rate  of  exit  of  aircraft 

Another  method  was,  therefore,  tried  in  order  to  estimate  the 
point  at  which  each  system  reached  its  limit  of  capacity.  Very  simply, 
this  method  recorded  the  time  the  first  "holding"  instruction  was 
given  during  each  trial.  The  NAFEC  team  of  observers  were  told  to 
record  this  time  in  a  format  similar  to  Table  11. 

The  idea  is  that  it  is  possible  to  estimate  the  relative 
sizes  of  beer  mugs  by.  .aeaenring  the  time  that  the  pourer  first 
stops  filling  the  mug,  and  withdraws  the  bottle.  This  is  not 
only  a  good  estimate  of  the  relative  size  of  the  mugs,  but  it 


interacts  vith  the  poursr’s  estimate  of  this  size;  erectly  vast 
ve  want  to  oeasure. 

In  general,  all  controller  teams  issised  their  first  instriJCtion  to 
hold  after  about  30  einutes  of  traffic  buildup.  It  vas  hoped  that  a 
sample  by  sample  comparison  of  the  exact  tiE>es  the  instruction  was  given 
would  reveal  a  pattern  sufficient,  for  making  an  objective  judgment 
regarding  controller  capacity. 

Team  I  illustrates  a  pattern  sufficient  to  conclude  tbat  the  ARTS 
system  became  overloaded  at  a  later  point  in  time  than  the  manual  system 
when  confronted  with  the  identical  traffic  sample.  With  sample  A,  team 
I  issued  their  first  'Tiold"  instruction  at  9:32.43  A.M.  using  the  manual 
system;  at  9:42.22  A.M.  using  ARTS.  With  sample  B,  this  same  team 
recorded  times  of  9;28.56  A.M.  with  manual;  9:37.47  A.M.  with  ARTS. 

With  sample  C,  the  times  were  9:35.09  .A.M.  manual;  9:36.55  ARTS.  With 
each  sample,  team  I  went  into  holding  at  a  later  time  when  using  the 
ARTS  system.  The  conclusion  would  follow  that  the  ARTS  system  resulted 
in  greater  controller  capacity,  if  this  pattern  were  repeated  for  all 
teams  or  even  a  majority  of  the  teams.  But,  the  pattern  for  the  other 
teams  was  too  variable  to  draw  any  such  conclusion.  The  average  time 
that  a  "hold"  instruction  was  first  given  for  all  teams  was  9:32.18  for 
manual,  9:33.18  for  ARTS.  Again,  this  is  inconclusive  evidence  of 
increased  capacity. 


11 

55FSC  csarmc&iZD  ccssrsonzas 

TIME  AT  fi3IC3  CCe?Z5fflJgRS  SLCB?  GS.  STOP  THE  AHHI7AL  FlOff 


H-ASOAL  SYSiai 


09:32:43 

09:28:56 

09:35:09 

09:26:05 

09:24:50 

09:24:25 

09:30:56 

09:26:33 

09:26:08 

1 

09:41:25 

09:41:35 

09:36:15 

09:34:00 

09:38:19 

09:25:15 

09:31:43 

09:38:46 

09:39:10 

09:42:22 

09:37:47 

09:16:29 

09:28:14 

09:21:05 

09:38:40 

09:39:00 

09:33:42 

09:26:27 

09:31:35 

09:33:40 

09:38:33 

W/  . 
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Sosever,  it  is  isterestiris  to  Bote  tbst  oa  tbs  71st  rca  oct 
of  a  total  of  72  is  tbs  expsrisiesit,  tbs  trainee  ccstroller  workirg 
tbe  arrival  position  did  aot  isss^  a  "bold”  iasixoctioa  dmrirjg  tbe 
entire  bour  voile  usii^  Aad,  be  did  perfozs  to  tbe  averse 

of  bis  group  in  the  abilitj  to  avoid  cusflicts.  But,  this  event 
vas  not  repeated  and  is,  ti^refore,  an  isolated  example  only  of 
tbe  potential  for  increased  capacity  available  frco  ARTS. 

There  is  a  clear  tenptation  to  conclude,  therefore,  that  based  upon 
tlie  ezperinental  results  there  is  no  increase  in  controller  capacity 
demonstrated  by  tbe  ARTS  systea.  But,  this  conclusion  ignores  the 
previous  discussion  concerning  the  difficulty  in  evaluatii^  a  aulti- 
diaensional  system.  To  draw  conclusions  in  one  dicension  requires  that 
the  performance  in  all  the  other  dimensions  be  held  equal.  However, 
this  is  not  the  case  for  the  present  evaluation  of  controller  capacity. 
The  number  of  conflicts  were  not  equal.  This  is  similar  to  an  evaluation 
of  a  typist’s  performance  based  -nly  on  speed  without  a  consideration  of 
the  number  of  mistakes  that  are  made. 

Figure  5  — ^  indicates  the  time  each  system  was  in  a  conflict  of 
the  IFR  X  VFR  variety.  This  figure  was  devised  to  highlight  differences 
in  system  safety,  and  it  is  introduced  together  with  a  detailed  dis¬ 
cussion  in  the  section  on  'conflicts" offered  below.  But,  this  graph 
could  just  as  readily  be  interpreted  to  indicate  controller  capacity 
by  assuming  some  fixed  time  in  conflict,  and  then  comparing  the  levels 
of  traffic  handled  by  each  system.  For  example,  for  the  same  value  of 
two  minutes  in  conflict,  the  ARTS  system  can,  according  to  Figure  5, 
handle  approximately  82  aircraft  per  hour,*  the  manual  system  67  per 

9/  See  page  81- 


hssr.  Iba  coaclmioa  thst  t±era  is  eitrher;  1)  feaer  conflicts,  or  2} 
increased  capadtj  is  a  izatter  of  choice  that  depeads  oq  the  particolar 
facet  of  perforsaace  that  tl^  sttadj  wishes  to  highlight.  In  the  sane 
vay,  it  can  be  argned  that  although  the  nir^>er  of  IPR  tairgets  on  the 
controller's  radar  scope  at  any  instant  in  tise  shosed  no  difference  in 
the  mrSber  of  targets  being  worked,  the  cniacity  of  the  ARTS  systea  was. 


in  reality,  Esich  greater.  Both  systems  had  an  average  of  five  VFS 
targets  on  the  radar  scope  at  every  instant  of  tiae.  To  the  namtal 
system,  however,  these  targets  imparted  little  information,  but  these 
same  targets  equipped  with  Hcde  C  transponders  were  providing  the  ARTS 
controller  with  additional  relevant  information  that  required  his 
further  action.  In  effect,  then,  the  ARTS  controller  was  working  a 
total  of  17  targets  compared  to  the  manual  total  of  12. 

Figure  A.l  was  drawn  in  order  to  highlight  the  differences  in 
the  number  of  conflicts  that  result  from  the  fact  that  the  ARTS  coutroller 
is,  in  reality,  working  with  more  targets  known  to  him  and  is,  there¬ 
fore,  able  to  perform  with  fewer  conflicts.  For  example,  for  an 
identical  value  of  1.4  for  the  index  of  orderliness,  the  ARTS  system 
can  handle,  at  any  Instant,  an  Increased  number  of  unkncsm  targets  of 
the  sample  varieties  Included  in  the  experiment;  The  manual  eysttem  can 
handle  1.5  aircraft;  the  ARTS  system,  4.5  aircraft. 

It  would,  therefore,  not  be  accurate  or  fair  to  cori^u^  that  there 
are  no  differences  in  controller  capacity.  The  more  reasonable  conclusion 


10/  See  page  145 j  Appendix  A. 


is  tnat  th&  st^saj  clsose  to  deauastrate  differences  ia  tbe  inost  critics! 
of  tbs  perforsisnce  meascres,  safety-  For  this  reasoa,  tbs  easjerineat 
was  desig:^  to  tncorer  differences  in  conflicts,  under  controlled 
conditions.  Ibis  cccipronised  tbe  ability  to  discover  differences  in 
oti^r  psrforsancs  measures,  if  differences  in  controller  capacity  are 
considered  to  be  of  critical  irsortance  then  it  is  recocrsnded  that  a 
different  experinent,  one  that  has  been  designed  to  elicit  these  special 
differences,  be  performed . 

b)  Conflicts;  A  Measinre  of  Safety 

This  section  of  the  report  deals  with  those  findings  designated  as 
being  of  critical  importance,  prior  to  the  conduct  of  the  experiment.  A 
detailed  presentation  of  these  results  are  shown  in  the  following  tables 
12  through  15; 


Certificated  Controllers 

Table 

12 

Number  of  Conflicts  (IFR  x  VFR) 

Index  51 

Number  of  Conflicts  (IFR  x  IFR) 

Index  52 

Table 

13 

Time  in  Conflict  (IFR  x  VFR) 

Index  54 

Time  in  Conflict  (IFR  x  VFR) 

Index  55 

Noncertificated  Controllers 

Table 

lA 

Number  of  Conflicts  (IFR  x  VFR) 

Index  51 

Number  of  Conflicts  (IFR  x  IFR) 

Index  52 

Table 

15 

Time  in  Conflict  (IFR  x  VFR) 

Index  54 

Time  in  Conflict  (IFR  x  IFR) 


Index  55 


above  tables  perrait  a  sample  saiiaple  coroarison  (aatched 
pairs  of  observatiotis)  for  each  teas  usisg  the  ARTS  and  nancal 
S3rStens. 


The  statistical  analyses  of  variance  perforced  on  these 
data  attei^ted  to  deteraine  whether  the  observed  differences 
in  conflicts  could  be  explained  as  randoa  variations,  or  were 
due  to  inherent  differences  in  the  systeas  theaselves.  In 
effect,  the  analyses  nade  statistical  adjustments  to  eliminate 
those  differences  due  to:  1)  variations  in  the  three  sample 
patterns  of  traffic,  2)  variations  in  the  six  different  teams  of 
controllers,  3)  variations  in  the  interaction  effects  of 
specific  controller  teams  operating  with  given  samples  of 
traffic.  The  remaining  variation,  or  experimental  error,  was 
used  to  determine  whether  observed  differences  5.n  system 
performance  could  be  judged,  objectively,  to  be  due  to  some  non- 
random  effect.  But,  these  analyses,  although  superior  in  statis¬ 
tical  power,  do  not  lend  themselves  to  intuitive  judgment  and 
insight.  The  mind  boggles  in  its  attempt  to  do  statistical 
analyses  of  variance.  For  this  reason,  a  simple  match-xq)  of 
results  is  presented  in  order  to  provide  some  intuitive  feelings 
for  how  the  systems  compare  when  the  same  team  controls  the 
identical  traffic  sample.  These  comparisons  are  amenable  to 
objective  judgments  to  any  stated  degree  of  statistical 
reliability,  but,  to  repeat,  the  statistical  power  of  these 
judgments  are  less  than  what  is  provided  by  the  more  sophis¬ 
ticated  analyses  of  variance  previously  reported  in  Table  8A. 
However,  the  results  shown  in  Table  8A  are  less  informative 
than  the  direct  match-up  of  observations.  For  illustrative 
purposes,  tlien,  the  detailed  comparisons  of  all  the  experi¬ 
mental  data  dealing  with  conflicts  are  shown  in  Tables  12 
through  15. 


Each  (+)  sign  indicates  that  there  were  more  violations  under 
the  manual  system  than  ARTS;  a  (-)  sign  indicates  the  reverse:  that 
ARTS  had  more  violations  than  manual.  A  (0)  indicates  a  tie  score. 
The  total  of  "(+;  0)"mu3t  add  to  18  for  each  index.  By  comparing 
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Table  j2 

Indices  51;  52 

IrtlHBER  OF  COKFIICTS 

HAFEC  Certificated  Controllers 
Sasple  by  Sarnie  Coaparison 
"Sign  Test" 


(18  Gsatcbed  coaparisons) 
Manual  (H)  vs.  ARTS  (A) 


Team 

1st  Half  Hour 

2nd  Half  Hour 

IFR-VFR 

IFR-IFR 

IFR-VFR 

IFR-IFR 

Index  51 

Index  52 

Index  51 

Index  52 

(M) 

(A) 

Sign 

(H) 

(A) 

Sign 

(M) 

(A) 

Sign 

(H) 

(A) 

Sign 

1 

a 

0 

0 

0 

2 

0 

-J- 

1 

3 

* 

6 

8 

- 

b 

0 

0 

3 

2 

"h 

2 

2 

0 

3 

6 

- 

c 

0 

0 

0 

0 

0 

3 

2 

+ 

8 

8 

e 

2 

a 

2 

0 

"h 

0 

0 

0 

0 

0 

0 

1 

1 

0 

b 

0 

0 

0 

1 

1 

0 

0 

1 

- 

0 

0 

0 

c 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4 

0 

+ 

3 

a 

2 

2 

0 

2 

0 

+ 

0 

0 

0 

6 

2 

+ 

b 

1 

1 

0 

0 

0 

0 

2 

2 

0  ! 

5 

7 

- 

c 

0 

0 

0 

0 

0 

0 

2 

0 

+ 

4 

2 

+ 

4 

a 

1 

0 

+ 

1 

1 

0 

3 

1 

2 

+ 

7 

1 

+ 

b 

0 

2 

- 

1 

5 

- 

2 

2 

0 

4 

4 

0 

c 

0 

0 

0 

2 

0 

+ 

1 

1 

0 

1 

6 

- 

5 

a 

1 

2 

«. 

2 

1 

+ 

2 

1 

+ 

3 

3 

0 

b 

1 

1 

0 

0 

2 

- 

2 

0 

+ 

2 

2 

0 

c 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

2 

- 

6 

a 

1 

0 

+ 

1 

0 

+ 

4 

1 

+ 

6 

3 

+ 

b 

0 

0 

0 

2 

2 

0 

2 

3 

- 

5 

4 

+ 

c 

0 

0 

n 

vr 

0 

0 

0 

1 

0 

+ 

7 

5 

+ 

Summary 

3+ 

6+ 

7+ 

7+ 

Totals 

2- 

2- 

3- 

5- 

+  =  more  conflicts  under  manual 


-  =  more  conflicts  under  ARTS 
0  =  same  number  of  conflicts 


Summary 

Totals 


Table  13 
Indices  54;  55 
TIME  IN  CONFLICT  (SECONDS) 
NAFEC  Certificated  Controllers 


Sample  by  Sample  Comparison 
"Sign  Test" 

(18  matched  con^tarisons) 
Manual  (M)  vs.  ARTS  (A) 


1 

I  1st  Half  Hour 

IFR-VFR 

IFR-IFR 

Index  54 

Index 

55 

(M) 

(A) 

Sign 

CM) 

(A) 

Sign 

0 

0 

0 

48 

0 

+ 

0 

0 

0 

128 

38 

+ 

0 

0 

0 

0 

0 

0 

310 

0 

+ 

0 

0 

0 

0 

0 

0 

24 

34 

- 

0 

0 

0 

0 

0 

0 

208 

220 

74 

0 

+ 

55 

5 

+ 

0 

0 

0 

0 

0 

0 

0 

0 

0 

41 

0 

+ 

14 

24 

0 

112 

- 

19 

111 

- 

0 

0 

0 

34 

0 

+ 

265 

195 

+ 

228 

26 

+ 

75 

41 

+ 

0 

58 

- 

0 

0 

0 

0 

0 

0 

202 

0 

+ 

51 

0 

+ 

0 

0 

0 

43 

51 

- 

0 

0 

0 

0 

0 

0 

6+ 

6+ 

2- 

5- 

more  time  in  conflict  under  manual 
more  time  in  conflict  under  ARTS 

same  time  in  conflict 


2nd  Half  Hour 


IFR-VFR 
Index  54 


57  124  “ 

127  100  + 

172  140  + 

0  0  0 

0  59  - 

0  0  0 

0  0  0 

107  97  + 

78  0  + 

155  95  + 

122  70  + 

125  61  + 

102  41  + 

26  0  + 

0  0  0 

255  58  + 

120  196  - 

23  0  + 


IFR-IFR 
Index  55 


389  417  - 

60  332  - 

382  404  - 

170  42  + 

0  0  0 

j  102  0  + 

305  128  + 

205  398  - 

116  91  + 

195  23  -J- 

78  126  - 

22  239 

100  225  - 

61  46  + 

23  61  - 

186  167  + 

204  267  - 

589  314  + 


73  - 


Table  14 

Indices  51;  52 

NUMBER  OF  CONFLICTS 

Noncertificated  Controllers 
Sample  by  Sample  Comparison 
"Sign  Test" 


(18  matched  comparisons) 
Manual  (M)  vs,  ARTS  (A) 


Team 

Sample 

1st  Half  Hour 

1  2nd  Half  Hour 

IFR-VFR 

IFR-IFR 

IFR-VFR 

IFR-IFR 

Index  51 

Index  52 

Index  51 

1  Index 

52 

(M) 

(A) 

Sign 

(M) 

(A) 

Sign 

(M) 

(A) 

Sign 

(M) 

(A) 

Sign 

1 

a 

1 

1 

0 

4 

3 

+ 

1 

2 

- 

4 

2 

+ 

b 

0 

0 

0 

1 

0 

0 

0 

0 

8 

1 

+ 

c 

0 

0 

0 

1 

0 

+ 

3 

1 

+ 

6 

4 

+ 

2 

a 

0 

0 

0 

3 

1 

+ 

1 

3 

• 

6 

1 

+ 

b 

0 

0 

0 

2 

2 

0 

1 

1 

0 

4 

2 

+ 

c 

0 

0 

0 

2 

0 

+ 

3 

2 

+ 

6 

0 

+ 

3 

a 

0 

0 

0 

2 

0 

+ 

1 

6 

m 

3 

3 

0 

b 

0 

0 

0 

3 

1 

+ 

2 

2 

1 

3 

- 

c 

0 

0 

0 

2 

0 

+ 

2 

1 

+ 

4 

2 

h 

4 

a 

0 

0 

0 

0 

1 

0 

0 

0 

1 

1 

0 

b 

0 

0 

0 

0 

0 

0 

1 

2 

- 

3 

2 

+ 

c 

0 

0 

0 

0 

0 

0 

2 

2 

0 

4 

2 

+ 

5 

a 

0 

0 

0 

1 

0 

+ 

1 

2 

6 

2 

+ 

b 

0 

0 

0 

1 

1 

n 

■■ 

0 

A 

5 

- 

c 

0 

0 

0 

0 

1 

- 

10 

3 

+ 

6 

a 

0 

0 

0 

1 

1 

H 

■1 

1 

0 

+ 

b 

0 

0 

0 

0 

2 

- 

2 

2 

0 

0 

0 

0 

c 

0 

0 

0 

0 

1 

- 

2 

3 

- 

2 

1 

+ 

Summary 

0 

9+ 

3+ 

13+ 

Totals 

0 

4- 

7- 

+  =  more  conflicts  under  manual 
-  =  more  conflicts  under  ARTS 
0  =  same  number  of  conflicts 
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Table  ^ 

Indices  54;  55 

TIME  IN  CONFLICT  (SECONDS) 

Noncertificated  Controllers 
Sample  by  Sample  Comparison 
"Sign  Test" 

(18  matched  comparisons) 
Manual  (M)  vs.  ARTS  (A) 


Team 

■ 

1st  Half  Hour 

2nd  Half  Hour 

IFR-VFR 

IFR-IFR 

IFR-VFR 

IFR-IFR 

Index  54 

Index  55 

Index  54 

Index  55 

(M) 

(A) 

Sign 

(A) 

Sign 

(M) 

(A) 

Sign 

(M) 

(A) 

Sign 

1 

a 

182 

251 

343 

195 

+ 

98 

76 

231 

42 

+ 

b 

0 

0 

0 

78 

0 

+ 

0 

0 

0 

576 

118 

0 

0 

0 

57 

0 

+ 

226 

101 

+ 

584 

269 

+ 

2 

0 

0 

0 

109 

68 

+ 

34 

165 

617 

30 

+ 

0 

0 

0 

224 

14 

+ 

15 

64 

- 

462 

65 

+ 

0 

0 

0 

73 

0 

+ 

42 

59 

- 

509 

0 

+ 

3 

0 

0 

0 

19 

0 

+ 

63 

237 

. 

100 

227 

• 

0 

0 

0 

90 

28 

+ 

92 

98 

- 

129 

205 

- 

0 

0 

0 

43 

0 

+ 

84 

110 

- 

242 

137 

+ 

4 

0 

0 

0 

0 

17 

0 

0 

0 

26 

14 

+ 

0 

0 

0 

0 

0 

0 

10 

99 

- 

143 

118 

+ 

0 

0 

0 

0 

0 

0 

109 

77 

+ 

806 

97 

+ 

5 

a 

0 

0 

0 

60 

0 

+ 

44 

126 

• 

777 

403 

+ 

0 

0 

0 

68 

51 

+ 

54 

69 

- 

155 

242 

- 

0 

0 

0 

0 

80 

- 

0 

52 

- 

813 

586 

+ 

6 

0 

0 

0 

3 

11 

m 

44 

21 

17 

0 

+ 

0 

0 

0 

0 

176 

- 

217 

90 

+ 
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0 

0 

c 

0 

0 

0 

0 

77 

- 

38 

126 

188 

18 

+ 

Summary 

11+ 

144- 

Totals 

) 

! 

1- 

5- 

12- 

3- 

+  “  more  time  in  conflict  under  manual 


-  =  more  time  in  conflict  under  ARTS 
0  =  same  time  in  conflict 


Kiae  test  results  asd  affixing  tbe  appropriate  sign  to  the  difference 
in  resslts,  ue  can  csoipaxe  sunaary  totals: 

T^le  12  provides  the  details  and  "sign"  comparison  for  the  number 
of  COTsflicts  of  both  the  IFR  x  IFR  (index  no.  52)  and  IFR  x  VFR  (index 
m.  31)  varieties  coDsiitted  by  the  H&FEC  team  of  certificated  con¬ 
trollers.  Sote  that  all  cases  favor  ABTS.  There  were  fewer  direct 
conparisoQS  in  fdiich  AfiTS  yielded  more  nosers  of  conflicts  than  the 
imgigal,  systea^for  both  halves  the  hour  and  for  both  varieties  of 
cosflictr. 

Table  13  which  depicts  the  time  each  system  was  in  conflict  tells 
the  story  essential  to  the  study  and  upon  idiich  the  majority  of  the 
doil<sr  benefits  claizaed  for  ARTS  III  rests.  There  is  little  difference 
in  the  ARTS  and  aanual  systems  ability  to  do  the  primary  job  of 
^paratic^  IFR  aircraft  (index  no.  55).  The  essential  difference  is 
that  the  ARTS  system  was  able  to  provide  additional  safety  in  the 
separation  of  ’L.RxWR  aircraft  (index  no.  54)  without  changing  any 
of  toary's  standards  and  procedures  for  controlling  aircraft  ir  the 
temiml  area. 

In  the  opinion  of  the  writer,  this  demonstration  of  reductions 
i.  conflicts  —  measured  in  pl^sical  tmits,  not  dollars  —  for 
certificated  controllers  using  ARTS  equipment,  is  sufficient  to 


Justify  its  installation  at  the  terminal  locations  identified  in  the 
third  lot  buy. 

Tables  14  aixl  15  are  similar  to  Tables  12  and  13  except  that  the 
comparisons  shown  are  for  the  trainee  controller  groups.  Table  14} 


11/  This  statement,  of  course,  did  not  appear  in  the  previous  draft 
version  of  the  study  forwarded  to  the  Office  of  the  Secretary 
of  Transportation  and  to  the  Office  of  Management  and  Budget; 
offices  that,  in  conjunction  with  the  Federal  Aviation  Admin¬ 
istration,  were  charged  with  the  responsibility  of  nudcing  a 
rational  decision  with  regard  to  ARTS  III.  It  has  been  more  than 
a  year  since  the  recommendation  of  the  FAA  was  confirmed  and  a 
decision  made  to  exercise  the  option  to  purchase  the  remaining 
third  lot  of  this  equipment.  There  is  no  way  of  knowing  what, 
if  at^,  influence  this  study  had  on  this  decision.  However,  there 
is  every  reason  to  suppose  that  the  decision  in  favor  of  ARTS  was 
made  on  the  basis  of  the  “issues,"  and  that  the  evidence  in  support 
of  the  argument  that  there  was  a  likelihood  of  increased  safety 
due  to  ARTS  III  was  brought  to  the  attention  of  the  decision 
makers,  and  discussed  in  detail.  The  reason  for  the  bold  asser¬ 
tion  made  above  concerning  the  superiority  of  ARTS  equipment  is 
to  remind  the  reader  that  the  sass  kinds  of  strong  assertions, 
based  in  major  part  on  subjective  judgments  only,  had  to  have  been 
made  by  those  charged  with  making  an  appropriate  decision  regarding 
ARTS.  Somehow,  a  delusion  seems  to  h»ve  taken  hold  of  the 
suppliers  of  analytical  studies  and  Ir  too  willing  customers 
that  it  is  possible  to  make  investiB.  decisions  solely  on  the 
basis  of  quantitative  evidence  gathered  under  the  heady  and 
objective  banner  of  cost/benefit  analysis.  To  this  point  we 
have  not  yet  assigned  dollar  values  to  our  study  results  and, 
therefore,  have  not  yet  entered  the  more  highly  subjective  world 
of  cost/benrfit  analyses.  ^  have  de^lt  only  with  objective  data, 
but  the  reader  ^s  again  reminded  that  despite  the  balanced 
e^>erimental  design,  the  statistical  controls  and  the  analyses 
of  variance,  subjective  judgments  were  required  at  every  critical 
jxmctmre  in  the  analysis.  Quantitative  studies  of  even  the  most 
elegant  sort  do  not  take  the  decision  maker  out  of  the  loop. 

They  merely  define  more  narrowly  the  areas  in  which  he  must 
assert  his  prejudices. 


J 
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1 

.  I 

index  S2,  confirms  that  the  trainee  controllers  using  ARTS  were  able  to 

( 

f  •  *  ' 

separate  IFR  x  IFR  airciraft  to  within  the  standard 'set  by  the  certificated 

i  ! 

controller  after  a  briefing  sfsssion  that  included  only  four  hours  of 
instruction;  just  two  hours  were  with  hands  -on  the  ARTS  simulator  equip¬ 
ment.  However,  index  51  shown  in  Table  14  also  indicates  that  the 

I  : 

trainee  did  not  do  nearly  as  well  in  separating  IFR  x  VFR  traffic, 

using  ARTS.  .This  important  category  was  the  single  indicator  in  which 

!  1 

the  trainee's  performance  did  not  match,  that  of  the  certificated  con- 
troller.  Perhaps,  this  group  was  under  greater' stress.  Somewhat 
su^risingly,  there  was  no  deterioration  in  the  trainee  group  performance 
in:  1)  operations  count  2)  controller  capacity  (as  measured  by  the  time 

I 

a  "holding"  instruction  was  first  given),'  3)  delays  and  time  in  system. 
But,  the  trainee  group  using  ARTS  evidently  could  not  perform  their 
ancilla^  job  of  vectoring  aircraft,  on  a  time  avai.lable  basis,  in  order  to 
avoid  conflicts  with  known  VFR  aircraft.  More  importantly,  as  Table  15, 
index  55  indicates,  the  trainee  controller  could  not  operate  the 
manual  system  satisfactorily  iu  the  primary  job  of  sepai  ating  IFR 
traffic.  Not  only  were  there  14  trials  out  of  18  in  which  the  ARTS 
system  resulted  in  less  time  in  conflict,  but  a  simple  arithmetic 
calculation  reveals  that  the  manual  system. was  in  conflict  for  a  total 
of  6375  seconds  during  the  second  half-hour  of  the  experiment  (com¬ 
pared  to  2571  for  ARTS).  This  amounts  to  an  average  of  354  seconds  for 
a  trial  period  of  30  minutes  (1800  seconds);  a  conflict  rate  of 
20  percent-  This  unsatisfactory  rate  for  the  mantzal  system  compared 
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to  a  statistically  significant  reduced  rate  for  ARTS,  a  rate  con^arable 
to  that  achieved  by  the  certificated  controller,  provided  the  basis  for 
the  modest  savings  in  training  costs  claimed  for  the  ARTS  system.  These 
dollar  benefits  provided  by  the  possibility  for  a  reduction  in  training 
time  can  be  disallowed,  however,  without  having  any  appreciable  effect 
on  the  study's  conclusion  to  install  ARTS  at  the  29  candidate  terminal 
locations. 

Figures  4-7  are  graphical  representations  of  these  conflict  data 
plotted  against  the  hourly  rates  of  airborne  traffic  generated  in  the 
experiment.  The  choice  of  the  appropriate  t-raffic  level  to  relate  to  a 
given  index  of  performance  depends,  of  course,  on  the  purpose  of  the 
illustration,  and  the  question  at  hand.  For  example,  when  attempting  to 
determine  dollar  benefits  and  costs,  the  appropriate  traffic  activity 
level  to  consider  would  be  the  average  annual  volume  of  traffic.  If, 
however,  the  problem  were  one  of  establishing  the  criteria  for  system 
design,  a  more  appropriate  measure  would  be  the  peak  levels  to  be 
handled  by  the  system.  On  the  other  hand,  for  purposes  of  establishing 
the  realism  and  validity  of  the  simulated  experiment,  the  more  appro¬ 
priate  activity  level  would  be  the  number  of  targets  that  were  con¬ 
trolled  at  any  instant  of  time.  This  latter  measure  of  traffic 
activity  provides  an  indication  of  the  workload  imposed  on  the  con¬ 
troller,  and  is,  in  fact,  used  in  this  study  for  the  purpose  of  evaluating 
the  realism  of  the  experiment;  described  below  in  section  E. 


since  a  further  and  detailed  use  of  the  data  for  conflicts  will 
bo  ^lied  upon  for  the  analysis  of  the  costs  and  benefits  of  the 
candidate  systems,  these  conflict  data  are  graphically  Illustrated 
below  In  relation  to  hourly  rates  of  traffic.  These  rates  will  be 
projected  to  average  annual  volumes  when  costs  and  benefits  are 
estimated  In  section  F. 

The  reader  Is  reminded  that  these  graphs  were  drawn  on  the  basis 
of  two  points,  determined  by  experiment,  for  each  system:  one  point 
for  each  half  hour  period  of  the  experiment:  f)  for  first  half;  s)  for 
second  half;  C)  for  combined  hour.  The  dotted  line  connecting  these 
points  is  a  postulated  relationship  only. 

Figure  4  -  (index  52  for  certificated  controllers)  shows  the 
indicated  difference  in  number  of  IFR  conflicts,  in  favor  of  ARTS. 

Figure  5  -  (index  54  for  certificated  controllers)  shows  the 
statistically  significant  reduction  in  IFR  x  VFR  time  in  conflict, 
in  favor  of  ARTS. 

Figure  6  -  (index  52  for  noncertif icated  controllers)  shows  the 
statistically  significant  reduction  i.i  number  of  IFR  conflicts,  in 
favor  of  ARTS. 

Figure  7  -  (index  55  for  noncertif icated  controllers)  shows  the 
statistically  significant  reduction  in  IFR  time  in  conflict,  in  favor 
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FIGURE  5 


TIME  IN  CONFLICT-IFR/VFR-MINUTES  PER  SIMULATION  HOUR 

VS 
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R6URE6 
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FIGURE  7 

Tir«1E  IN  CONFLICT  -IFR/IFR-  MINUTES  PER  SIMULATION  HOUR 
'  VS 

HOURLY  IFR  TRAFFIC 
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E.  The  Application  of  Test  Results  to  the  Real  World 


A  critique  of: 

1.  The  Technique  of  Synaciic  Simulations 
II.  The  Traffic  Sample 
I.  Dynamic  Simulations 

An  explicit  study  requirement  was  the  determination  of  the 
criteria  for  installing  ARTS.  It  was  considered  not  sufficient  to 
demonstrate  that  ARTS  is  a  su^^erior  system.  The  conditions  for  which  this 
superiority  could  be  expected  to  hold  had  to  be  specified  as  well. 

These  conditions,  the  criteria  for  installation,  are  usually  defined  by 
some  measure  of  traffic  activity.  Study  methods  were  limited,  there¬ 
fore,  to  those  in  which  it  was  possible  to  balance  and  control  patterns 
of  traffic  to  any  given  level  of  activity.  The  only  methods  that  qualify 
are  those  employing  abstractions  or  simulations  of  reality.  But,  the 
resort  to  an  abstraction  of  reality  does  not  necessarily  compromise  the 
validity  of  a  study.  The  statement  that  "reality  is  difficult  to  under¬ 
stand"  is  a  platitude  that  doesn't  bear  repeating,  but  it  is  a  relevant 
comment  nonetheless.  For  exaiiq>le,  a  real  world  experiment  took  place 
at  Knoxville,  Tennessee,  during  the  latter  part  of  1970.  Its  purpose 
was  to  compare  the  communications  workload  for  the  ARTS  and  manual 
systems.  But,  since  traffic  levels  and  other  variables  of  reality  could 
not  be  controlled,  this  experiment  was  not  able  to  discern  differences 
in  these  workloads;  differences  that  were  apparent  at  high  levels  of 
statistical  significance  from  a  dynamic  simulation. 
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All  study  methods  face  the  single  and  identical  challenge  of  being 

able  to  predict  results.  There  is  no  scientific  responsibility  to  prove 

12/ 

that  an  experimental  setting  is  realistic.  —  But,  the  dynamic  simu¬ 
lation  method  of  investigation  is  quite  able  to  meet  this  superfluous 
challenge.  By  meeting  this  challenge  it  is  hoped  that  we  may  increase 
the  study's  believability,  if  not  its  scientism. 

Every  effort  was  made  to  virtually  reconstruct  the  setting  of  men 
and  equipment  in  a  typical  terminal  area.  Mechanical  devices,  target 
displays  and  symbology  were  designed  to  be  as  close  a  representation  of 
the  actual  systems  as  the  art  of  dynamic  simulations  would  allow .^^^he 
anxiety  and  stress  of  controlling  live  traffic  could  not,  of  course,  be 
duplicated,  but  the  experiment  was  not  free  of  stress  and  did  not 
invol’^’e  frivolous  controllers  working  in  a  relaxed  setting.  Yet,  the 
view  persists  that  results  obtained  from  dynamic  simulations  are 
"tainted"  because  they  are  derived  from  artifically  created  "play 
settings." 

The  world  of  quantitative  analyses  abounds  in  abstractions  of 
reality  based  on  regression  analyses,  computer  generated  simulations  and 
other  mathematical  models  with  links  to  reality  that  are  frequently  more 
tenuous  than  those  generated  by  attempts  to  simulate  an  environment. 

For  some  reason,  one  seldom  sees  a  requirement  imposed  on  these  more 
usual  and  acceptable  analytical  techniques  to  verify  their  method.  To 

12/  For  a  discussion  of  this  int  see  Friedman,  "Essays  in  Positive 

Economics,"  Ref.  6. 
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repeat,  the  verification  process  consists  solely  of  an  examination  of 
how  well  a  studj’  predicts  results,  but  memory  provides  few  examples  of 
attempts  by  studies  using  "acceptable"  techniques  to  confront  their 
conclusions  with  data  that  were  previously  unavailable.  However,  this 
study  will  attempt  to  meet  the  challenge  of  verifying  its  method  and 
Its  conclusions: 

1)  The  most  important  single  item  of  verification  is  the 
discovery  of  the  increased  probability  of  conflicts  resulting  from  the 
mixing  of  known  vs.  unknown  traffic.  This  problem  area  identified 
solely  on  the  basis  of  the  simulated  experiment  is  very  much  in  the 
news.  William  M,  Flener,  Director  of  the  FAA's  Air  Traffic  Service, 
called  the  problem  of  unknown  aircraft  the  most  important  one  facing 
today's  air  traffic  control  system.  —  This  same  conclusion  is  con¬ 
tained  in  the  Near  Midair  Collieiou  Report  of  If  >8.  Moreover, 

Congressional  Committees  and  other  advisory  groups  have  repeatedly 

14/ 

drawn  attention  to  this  problem.  —  The  independent  identification, 
but  more  importantly,  the  ability  to  quantify  a  problem  that  is  subse¬ 
quently  discovered  to  be  of  critical  concern  to  the  FAA  supports  the 
contention  that  the  simulated  experiment  was,  indeed,  realistic  and 
valid. 

13/  In  his  speech  "Air  Traffic  Control  of  Today"  delivered  before  the 
National  Aviation  System  Planning  Review  Conference  sponsored  by 
the  Department  of  Transportation,  1972. 

14/  The  Associated  Press  summary  of  the  National  Transportation  Safety 
Board  Report  on  Midair  Collisions,  released  March  1971,  is  shown 
on  the  next  page. 
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"181 

AERIAL  COLLISIONS 

BY  VERN  HAUGLAND 

WASHINGTON  (AP)-THE  NATIONAL  TRANSPORTATION  SAFETY  BOARD, 
AURttED  AT  THE  NATION'S  AERIAL  COLLISION  DEATH  TOLL,  URGED  A 
SISTER  AGENC-  TODAY  TO  TICHTEN  PILOT-QUALIFICATION  AND  AIRCRAFT- 
EQUIPMENT  RULES  AND  ACCELERATE  TRAFFIC-SEPARATION  PROCEDURES. 

IN  A  BULKY  REPORT  ON  A  1969  INQUIRY,  THE  SAFETY  BOARD  NOTED 
THAT  OF  223  SKY  COLLISIONS  IN  THE  10  YEARS  THROUGH  1968  ABOUT 
HALF  WERE  FATAL  AND  CAUSED  528  DEATHS. 

THE  VAST  MAJORITY  OF  THE  COLLISIONS  OCCURRED  IN  CLEAR 
WEATHER,  THE  BOARD  SAID  IN  URGING  CORRECTIVE  ACTION  BY  THE  FEDERAL 
AVIATION  AEMINISTRATION, 

ALTHOUGH  98  PER  CENT  OF  THE  COLLISIONS  INVOLVED  LIGHT  PLANES, 
AND  AIRLINE-TYPE  PLANES  WERE  INVOLVED  IN  ONLY  6v7  PER  CENT  OF  THE 
ACCIDENTS,  THE  OCCUPANTS  OF  THE  AIRLINERS  AMOUNTED  FOR  66  PER  CENT 
OF  THE  DEATHS,  IT  SAID. 

IF  THE  ACCIDENT  AND  FATALITY  RATES  REMAIN  THE  SAME  AND  IF 
TRAFFIC  INCREASES  AT  THE  FORECAST  RATE  THE  UNITED  STATES  MAY  HAVE 
335  ACCIDENTS  AND  792  DEATHS  IN  THE  NEXT  10  YEARS,  THE  BOARD  SAID. 

"NO  ONE  SOLUTION  WILL  ELIMINATE  ALL  MIDAIR  COLLISIONS,  BUT  MUCH 
CAN  BE  DONE  TODAY  TO  REDUCE  OR  ELIMINATE  THE  COLLISION  POTENTIAL, 

THE  BOARD  REPORTED. 

ACCORDINGLY,  IT  URGED  THAT  THE  FAA  ELIMINATE  THE  FREE  MIXING  OF 
"UNKNOWN"  AND  "KNOWN"  TRAFFIC— PLANES  OPERATING  UNDER  VISUAL 
FLIGHT  RULES  (VFR)  AND  THOSE  OPERATING  BY  INSTRUMENTS  (IFR)— 
ESPECIALLY  IN  TERMINAL  AREAS . 

KB641PES  MAR  1 
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b)  The  project  manager  for  the  experiment,  D.  0.  Brown  of 
NAFEC,  stated  that  he  would  have  no  hesitancy  in  certifying  16  out 

of  the  18  trainee  controllers  who  took  part  in  the  experiment.  These 
controllers  would  have  to  be  under  proper  supervision,  but  a  distinct 
advantage  of  the  ARTS  system  is  that  information  essential  to  the 
supervisory  function  is  readily  displayed  on  the  radar  scope.  The 
project  manager's  statement  was  used  as  part  of  the  data  confirming 
that  training  costs  can  be  reduced,  but  for  the  purpose  of  the 
argument  at  hand  it  indicates  that  a  controller  can  be  judged  on  the 
basis  of  his  performance  with  simulator  equipment.  Other  controllers 
have  confirmed  that  it  is  a  relatively  simple  matter  to  tell  whether 
someone  knows  his  "stuff"  by  watching  him  perform  in  the  simulated 
environment  employed  in  the  present  experiment. 

c)  The  prediction  that  training  costs  could  be  reduced  with 
ARTS  was  confirmed  by  an  unsolicited  statement  made  at  a  recent  meeting 
of  the  Air  Traffic  Controllers  Association.  — ^ 

d)  Finally,  the  results  of  the  simulation  were  discussed 
with  both  the  New  York  (Common  IFR  Room)  and  Atlanta  facilities. 

Judging  by  the  "we  told  you  so's"  one  wonders  why  we  needed  to  do  the 
experiment.  (To  quantify  their  impressions,  of  course).  Statements 

15/  M.  Flowers,  a  controller  at  the  Atlanta  terminal  facility,  remarked 
that  her  facility  found  that  it  is  easier  to  train  controllers  with 
ARTS.  This  remark  was  made  during  a  question  and  answer  session 
of  a  panel  discussion  of  .-dlTS  III  equipment  sponsored  by  the  Air 
Traffic  Controllers  Association  in  Denver,  Colorado,  October  1971. 
The  training  supervisor  at  the  Atlanta  facility  confirmed  these 
remarks . 
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made  by  these  facilities,  and  reported  by  the  Mfttional  Bureau  of 
Standards  — ^^ahout  the  a^^ility  to  train  controllers  more  easily  using 
ABTS  and  the  ability  to  reduce  commimications  workload  were  confirmed 
by  the  simulated  experiment.  Statements  concerning  increased  terminal 
capacity  made  by  individual  controllers  at  Atlanta  and  again  reported  by 
NBS  in  an  indq>endent  study  were- not  confirmed,  however.  But,’  both  of 
the  above  n-imed  terminals,  Ifew  York  and  Atlanta,  en5>loy  levels  of  ' 
automation  in  excess  of  ARTS  III.  '  '  : 

In  summary,  the  dynamic  simulation  employed  was  able  to:  1)  define 
the  criteria  of  installation  because,  it  could  statistically  control  for 

J 

levels  of  traffic  activity;  2)  quantity  differences  in  system  performance 
chat  were  not  discernable  from  real  world. data  that  could  lust  be 

I 

balanced  experimentally  or  controlled  statistically;  3)  demonstrate 

I 

,  { 

that  it  is  capable  of  predicting  and,  thus,  being  verified  by  results 
in  the  real  world. 

I 

It  is  reasonable  to  conclude,  therefore,  that  the  simulated 
environment  of  men  and  equipment  employed  at  provided  a  valid 

analytical  method  for  comparing  the  candidate  systems  of  air  traffic 
control.  An*,  like  all  methods  of  analyses,  its  use  is  limited  to 
those  investigations  for  which  its  validity  can  be  established. 


16/  "All  Assessment  of  the  Benefits  ol  ARTS  Based  on  Controller  Experience 
at  Atlanta  And  New  York,"  see  Ref.  7, 


$ 


II.  The  Traffic  Sample 

Another  important  consideration  in  deciding  whether  the  results 
from  the  simulated  world  can  be  transferred  to  the  real  worxd  is  a  com¬ 
parison  of  the  level  of  traffic  activity.  Were  the  aircraft-  types  and 
traffic  densities  eoDplcyed  in  the  simulation  comparable  to  the  levels 
that  might  be  expected  realistically? 

The  meastnre  of  traffic  activity  appropriate  to  the  question  of 
whether  results  from  a  simulation  are  applicable  to  the  real  world  is  the 
number  of  targets  that  must  be  controlled  at  any  instant  in  time. 

Reliable  statistics  for  this  measinre  of  activity  are  not  published  in  a 
form  that  categorizes  aircraft  sufficiently  for  our  purposes.  And  a 
direct  census  of  those  terminals  included  in  the  third  lot  buy  of  ARTS  III 
equipment  had  to  be  made,  letters  were  sent  to  six  of  these  terminals, 
selected  at  random,  asking  for  a  determination  of  the  peak  number  of 
targets  appearing  on  their  radar  scope  dinring  the  busy  hour  of  the  week 
in  which  the  letter  was  received.  In  addition,  these  terminals  were 
asked  to  further  identify  these  targets  as  "known"  or  "unknown"  according 
to  the  following  definition: 

ISiovn  aircraft  targets  are  those  in  which  spatial  information 
in  three  dimensions  is  available.  Aircraft  which  conform,  in  general, 
to  this  definition  include, 

a)  All  IFR  aircraft;  arrivals  and  departures 

b)  All  IFR  nircraft  "overs"  under  positive  control 


c)  All  VPR  aircraft  in  visual  contact  and  under  tower  control 


Unknown,  all  otners  includii^ 

d)  VFR  aircraft  not  under  visual  surveillance  of  tower 

e)  IPR  ''high  overs";  not  tmder  positive  control  of  tenainal. 
The  sum  of  categories  a)  through  0)  represents  the  total  radar  contacts 
reported.  These  are  shown  in  Table  16.  Data  for  the  years  1965-1970 
are  from  published  FAA  sources,  and  are  not  categorized  according  to 
"kn«»a"  and  "unknown."  Data  for  the  year  1971  are  the  responses  to  the 
direct  inquiry  made  to  six  candidate  terminals. 

Table  16  reveals  that  an  average  of  58  targets  were  counted  on  the 
radar  scopes  of  the  six  terminals  during  some  busy  instant  in  the  week  of 
March  8,  1971.  Of  this  nimber,  63  percent  were  unknown. 

Table  16 

PEAK  INSTANTAJIEODS  RADAR  CODITT  AT  SIX  CAin)inATE  TEHMINALS 


1971  Survey^,/ 


Total  Radar  Cotmti^  | 

1965 

1966 

1967 

1968 

1969 

19701/ 

48 

45 

61 

64 

65 

66 

58 

55 

56 

62 

70 

94 

22 

17 

24 

42 

58 

52 

64 

59 

87 

74 

71 

63 

40 

53 

48 

43 

54 

44 

63 

56 

57 

65 

68 

60 

1.  Albany 

2.  Buffalo 

3.  El  Paso 

4.  Portland,  Ore 

5.  Des  Moines 

6.  Tampa 


%  Known  _ 
Total 


SOURCE: 

\f  Airport  Activity:  Peak  Day;  Busy  Hour,  FAA,  1970 
2/  Response  to  letter,  March  8,  1971. 


Known 

Unknown 

17 

30 

20 

65 

13 

34 

39 

41 

25 

30 

1 

27  : 

45 

Table  17  depicts  the  world  of:  simulation.  In  this  world,  two 
systems  of  air  traffic  control  were  operating.  There  is,  therefore, 
a  distinction  between  "knowns"  and  "unknowns'*  according  to  the  system 
in  use: 

ARTS:  Known  consist  of, 

a)  All  IFR  aircraft.  There  were  no  IFR  "overs"  or  VFR 
operations  (takeoffs  or  landings)  in  the  simulation 

b)  All  VFR  overs  with  Mode  C. 

Therefore,  unknowns  are  all  VFR  without  Mode  C. 

Manxial:  Known  consist  of, 

a)  All  IFR  aircraft 

Therefore,  unknowns  are  all  VFR  aircraft. 


Table  17 

PEAK  INS  CANTAHEOUS  RADAR  CODNT  F(»  SIMDIATED  TERMINAL 


Arrivals 

Departures 

Overs 


IFR 

VFR 

ARTS 

9 

— 

Known  9 

3 

— 

3 

7* 

_5 

Total  Known:  18 

Grand  Total:  19 

*A11  VFR  aircraft  were  programmed  to  be  transponder  equipped. 

80  percent  of  these  VFR  aircraft  were  programmed  to  have  Mode  C 
transponders. 


MANUAL 

9 

3 

_0 

12 

19 
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it  is  evident  from  Table  17  thai  the  possibility  for  conflicts  is 
isuch  less  in  the  simulated  world:  I'i  simulated  airborne  targets  com¬ 
pared  to  a  peak  realistic  level  of  58  instantaneous  targets.  However, 
in  several  oral  present itions  made  of  this  study,  there  was  some 
criticism  voiced  over  the  fact  that  all  VFR  aircraft  in  the  simulated 
exercise  were  programmed  to  have  transponders;  an  average  of  6  aircraft 
each  instant.  And^.  80  percent  of  the  transponders  had  Mode  C  capability. 
In  effect,  the  study  was  criticized  becatise  it  employed  traffic  samples 
that  were  not  realistic.  The  present  fleet  of  aircraft  are  not  equipped 
with  this  high  a  proportion  of  transponders.  This  criticism  is  not  well 
founded,  however,  and  the  following  argtnnents  are  offered  to  support 
this  contention: 

An  AkTS  system  consists  of  complementary  components,  on  the  ground 
and  in  the  air,  capable  of  automatically  generating  a  block  of  data  on 
the  controller's  radar  scope.  An  airborne  transponder  is  an  integral 
part  of  the  ARTS  system.  Any  conflict  which  results  while  ARTS  is 
controlling  traffic  cannot,  therefore,  be  considered  as  an  ARTS  conflict 
unless  the  aircraft  involved  had  an  airborne  transponder. 

A  plausible  alternative  would  have  been  to  use  a  traffic  sample 
which  had,  say,  50  percent  of  the  VFR  aircraft  equipped  with  trans¬ 
ponders,  or  some  other  number  that  represented  a  realistic  forecast  of 
airborne  traffic.  Such  a  forecast  was  used  to  estimate  the  proportion 
of  transponder  equipped  IFR  traffic  employed  in  the  simulation.  This 
forecast  depends,  of  course,  on  the  FAA's  policy  regarding  transponders. 
Presently,  at  specified  terminal  locations  where  the  Terminal  Control 


Area  concept  is  in  use,  100  percent  of  the  aircraft  using  these  terminals 
must  be  transponder  equipped.  But,  the  adjusting  of  the  proportion 
assumed  for  transponder  in  order  to  ^ree  realistically  with  some  given 
forecast  would  not  add  to  the  study's  validity.  The  purpose  of  any 
study  is  to  leam  something,  and  the  experiment  was  desigited  ti?  provide 
the  maximum  amount  of  information  for  a  given  nuiid>er  of  trial  observa¬ 
tions  .  We  would,  undoubtedly,  leam  very  little  if  one  half  of  the  time 
the  ARTS  system's  ability  to  convert  unknown  traffic  into  known  w.is 
limited  by  the  fact  that  it  was  operating  in  a  manner  that  was  ideutical 
to  the  manual  system.  There  is  no  difference  between  systems  if  there  is 
no  airborne  transponder  transmitting  data  that  can  be  displayed  auto¬ 
matically  on  a  radar  scops.  Had  it  been  suggested  that  all  conflicts 
involving  non-transponder  equipped  aircraft  be  deleted  from  the  ARTS 
account  or  transferred  to  the  manual  account  there,  probably,  would  have 
been  no  criticism  voiced.  However,  it  was  the  announced  intention  of 
the  study  not  to  alter  any  data  generated  by  the  experiment.  But,  if 
such  a  transfer  of  data  were  allowed,  the  result  would  be  that  only 
those  conflicts  involving  aircraft  with  transponders  were  being  attributed 
to  ARTS.  In  effect,  this  is  precisely  what  was  accomplished  by  a  prior 
programming  of  the  traffic  sample.  An  80  percent  proportion  of  Mode  C 
transponders  was  used  in  the  hope  that  some  additional  analytical 
insight  would  result.  If  100  percent  of  the  airevaft  had  transponders 
capable  of  being  tracked  and  displaying  ground  speed  and  only  80  percent 
of  these  were  capable  of  providing  altitude  information,  it  was  thought 
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that  some  basis  could  be  developed  for  deciding  which  of  the  several 
features  of  ARTS  III  automation  were  most  effective  in  reducing  con¬ 
flicts.  This  information  is  not  essential  to  the  present  evaluation 
which  was  designed  to  compare  the  total  ARTS  III  package  of  automation 
with  the  manual  package.  But,  it  was  anticipated  that  stibsequent 
studies  might  be  required  in  the  near  future  in  order  to  decide  on  the 
most  effective  features  of  automation  to  purchase  for  installation  at 
the  smaller  sized  terminals.  The  evidence  concerning  the  features  that 
contribute  most  to  conflict  reduction  is  not  clear  and  is  reported 
elsewhere  (Appendix  A).  What  is  clear  is  that  ARTS  as  a  total  sv,stem 
of  automation  was  able  to  demonstrate  fewer  conflicts  for  traffic 
samples  that  differed,  on  the  average,  by  as  few  as  five  unkn'sr*.'  air¬ 
craft.  The  relevant  yardstick  for  judging  the  comparability  r,r  the 
/iRTS  vs.  the  simulated  world  is,  therefore,  not  the  percentage  of  air¬ 
craft  that  had  transponders,  but,  rather,  the  probabilif/  ‘chat  as  many 
as  five  unknown  or  VFR  aircraft  could  be  expected  at  any  instant  on  the 
radar  scope  of  a  candidate  ARTS  III  terminal.  Table  16  indicates  that 
this  probability  is  quite  high.  At  the  instant  shown,  there  was  an 
average  of  41  imknown  aircraft  at  all  of  the  candidate  locations  at 
which  measurements  were  taken.  The  conclusion  is  drawn  that  from  the 
view  of  both  total  numbers  of  instantaneous  airborne  targets,  and 
numbers  of  tmknown  targets,  the  simulated  world  employed  traffic  levels 
which  were  well  below  those  encountered  in  the  real  world.  Any  results 
which  depend  on  the  level  of  traffic  activity  used  in  the  simulation  are, 
therefore,  likely  to  occur  realistically. 
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Tables  16  and  17,  which  compare  total  numbers  of  known  and  unknown 
aircraft  within  the  terminal  area  also  suggest  that  the  current  criteria 
for  installing  ARTS  equipment  be  modified.  The  measure  of  traffic 
activity  'hat  is  more  appropriate  to  estimating  the  probability  of  a 
midair  collision  is  the  total  number  of  aircraft  within  the  terminal 
area,  and  not  merely  the  number  of  operations  at  the  primary  terminal. 
For  this  reason.  Table  18  has  been  amended  to  include  itinerant  opera¬ 
tions  at  all  secondary  airports  having  control  towars  within  the  juris¬ 
diction  of  the  primary  airport.  This  table  was  used  in  the  subsequent 
estimation  of  costs  and  benefits  attributable  to  ARTS  III.  Despite 
the  marked  increased  activity  ever  the  numbers  previously  shown  for 
operations  at  the  primary  airport  only,  shown  in  Table  2,  Table  18 
still  understates  the  measure  of  activity  that  is  most  appropriate  to 
a  determination  of  the  likelihood  of  a  midair  collision,  i.e.,  the 
total  number  of  aircraft  that  traverse  the  terminal  airspace. 

The  activity  levels  shown  in  Table  18  are  an  understateiiient 
because  they  do  not  include:  1)  military  operations  at  military 
bases  within  the  jurisdiction  of  the  primary  terminal;  2)  operations 
at  secondary  airports  without  control  towers;  3)  aircraft  vdiich  are 
flying  through  the  terminal  zone  and  which  do  not  originate  or 
terminate  as  an  operation  at  either  the  primary  airport  or  at  any 
of  the  secondary  airports  within  the  terminal  area;  4)  all  local 


traffic . 


ACTUAL  AND  FORECAST  ITINERAKT  TRAFFIC  AT  PRIMARY  AND  SECONDARY  AIRPORTS 
WITHIN  JURISDICTION  OF  THE  PROPOSED  ARTS  III  LOCATIONS  <3RD  LOT  PROCUREMENT) 


<0 


o 

o 

O 

o 

o 

o 

o 

o 

o 

o 

o 

o 

C4 

o\ 

o 

in 

in 

* 

* 

r 

CO 

00 

o 

IN. 

Ov 

in 

CO 

C>l 

CO 

C^1 

m 

o  o  o  o  o 

o 

o  o  o 

o  o  o 

o  o  o  o 

CO  O 

o 

o  o  o  o  o 

o 

o  o  o 

o  o  o 

o  o  o  o 

o  o 

o 

C4  O  CO  CJ 

*  A  «t  M 

00 

CO  O  CO 

4k  A 

ov  o\  oo 

4k  4k  4k 

GO  0\  v-4  CO 

4k  *  «k  «k 

o 

4k  4k 

4k 

o  o»  cn  o 

00  00  lO 

f-c  oo  O 

CO  o 

f-4  00 

o\ 

tr»  »-*  00  O'  -a- 
1*4 

fv 

-4‘  cu  CO 

0\  ^ 

f-*  CM 

CM  c^  CM  in 

f-4  00 

CJ' 

o 

o 

o 

o  o 

o 

o 

o 

o  o 

00  1 

1  f  CO 

rv  1  r** 

t 

1  1  A 

«k 

A  j  4k 

r« 

a\ 

4*i  tH 

Ov 

C\ 

<X\ 

vO  xo 

f-4 

tH 

O 

O 

O  O 

O 

O 

O  O 

r. 

1  rv 

00  1  00 

k  4k  1 

1  •* 

»  1  4k 

no 

o  o 

O 

o 

CO  CO 

CM 

CM 

*-4  *-4 

.611 

.798 

o 

«M 

r-. 

1 

Mi* 

rv 

CO 

rM 

't 

.440 

m 

o 

xD 

00 

St 

<»• 

o 

CO 

OO 

CO 

ox 

ON 

00 

XO 

O 

4k 

4k 

4k 

4k 

A 

4> 

CM 

CM 

00 

•A 

fv 

CO 

CM 

o 

CO 

CM 

CM 

CM 

CO 

CM 

o  «o  O  t«-  o 

O 

O  Sj- 

CO  xO  OX 

csi 

en  o  t<k 

00  CO  rH  m 

CM 

CO  in 

o  «n  m 

xO  O  >0  N 

CO  r-l  sj* 

CM  CM 

in 

<n  o  .3- 

0x0x0 

m  Ox  <7X  <t 

XO  XO  CM 

4k  4k  4k  4k  «k 

4k 

4k  4k  4k 

4k  4k  4k 

4k  4k  4k  4k 

A  4k  4^ 

xo  m  o  no 

o 

O  1^ 

CO  VH  OX 

CO  \o  O  <-4 

f-l  O  CM 

^  \o  r-  CM 

o 

in  ON 

ox  CM  t-C 

in  CO  XO 

CO  Ox  C*4 

CM 

rM 

rM 

r*.  rv 

in 

XO  xO 

rH  rM 

C^  Cx 

00 

!*-► 

CM  CM 

rM  ?-C 

in  m 

1  1  1  f>k 

CM 

^  I  T-C 

Sj-  I  s^ 

1  t 

ox  1  ox 

4k  1  1  1  4k 

4k 

4k  1  4b 

4k  1  4k 

•  1  1  * 

M  1  ^ 

xO  XO 

CM 

rv 

xO  xO 

r-4 

CO 

CM  CM 

ox  Ox 

C^  OX 

f-M  rH 

*-< 

rH  rM 

1-C  rM 

a 

4*^ 

fH 

o 

rM  4J 

xU 

CQ 

-  xU 

G 

G  •H 

M 

< 

O  C  ,G 

/-N 

«U  M  S 

p>4 

d 

G 

c 

0 

0 

PO 

O  00  00 

4G 

•o 

CO 

^  U  U 

CO 

G 

<; 

C4  0  d 

•CJ 

V4 

0) 

rH  ^  ^ 

G 

4J 

•  4 

•  PQ  CO  CO  iH 

0 

4J 

C  0  rH 

CQ 

4J  u  U  ^ 

•H 

4 

0 

Cd 

«d 

Cd 

GOG 

G  <0  O  4)  xU 

U 

©  *0 

>4 

u 

Ph  u 

Ox  S  -H 

< 

IH  ^  JJ  4J  o 

u  c 

U 

0 

0  <  c 

1 

G  0 

O  q)  0)  H 

0 

0  a  H 

•o  5-1 

iJ 

G  0  Eh 

JJ 

0)  CD  pL4  (I4 

tH 

CO 

G 

fG 

0  *J 

G«  cd 

0 

*  4J 

4H 

4  0 

00  U  c 

9 

B  u  *  * 

u 

*0  ^1 

rH 

»H  0 

*H 

X4H 

G  cd  xi  4-1 

0 

G  0 

•rl 

0  M  Cl 

M 

a  »■< 

UH 

H  CO  CO  CO 

s 

a  pu  s 

73  0  5: 

G  :5 

0  0 

0 

-rt 

fH 

C 

C 

4 

AJ 

<d 

•u 

fH 

(4 

rH 

;>x 

4 

G 

0 

W 

3 

B 

SQ 

Pu 

0 

Q 

0 

ACTUAL  AND  FORECAST  ITINERANT  TRAFFIC  AT  PRIMARY  AND  JECONDARY  AIRPORTS 
WITHIN  JURISDICTION  OF  THE  PROPOSED  ARTS  III  LOCATIONS  (3RD  LOT  PROCUREMENT) 


CO 

VO 

o 

o 

O 

O 

o 

- ?5 — 

o 

o 

O 

o 

O 

o 

o 

CO 

o 

eg 

o 

Ov 

VO 

CO 

A 

eg 

A 

A 

A 

A 

A 

to 

A 

CO 

eg 

VO 

CO 

VO 

o 

eg 

OV 

00 

o 

CO 

oo 

rH 

tH 

A 

C»v 

o  o  o 

o  o  o 

o  o  o  o 

O 

o 

o  o  o  o 

o  o  o 

o  o  o 

o  o  o 

o  o  o  o 

O 

o 

o  o  o  o 

o  o  o 

O  04  N 

vO  O  vD 

m  ov  r-( 

C^ 

rH 

ov  eg  VO 

eg  t-g  CO 

A  A 

A  A  A  A 

A 

A 

A  A  A  A 

AAA 

M  r-l  rt 

cO  o 

CO  OO  sf  O 

CO 

OV 

rH  irt  00 

O  Ov  OV 

0\  0^ 

O  00  ov 

CM  eo  in 

oo 

CO 

O  O  VO  VO 

gj*  VO  O 

rH  f-4 

r-4 

eg  VO  00 

iH  CM 

o  o 

o  o 

o  t  o 

«k  I  * 

o  o 

00  00 


o  o 

o  o 

I 

M  I  ^ 


o  o  o 

o  o  o 

00  I  f  CO 

«»  I  t  *  A 

m 

!>.  O 


o  o 

o  o 

\0  I  I 
I  I  •» 

vO  ^ 

o\  o\ 


o  o 

o  o 

a\  I  Q\ 


®  Pi 
•H  fn 

U  M 

o 

o* 

eg 

00 

rH 

eg 

ov 

^S  fl  tH 

vO 

to 

VO 

eg 

CO 

IV. 

eg 

U  ® 

to 

CO 

CO 

vO 

eg 

4J 

« 

• 

• 

• 

>4  O 

o 

rH 

to 

o 

rv. 

00 

O 

VO 

CO 

fH 

VO 

CO 

Ov 

CO 

eg 

o 

eg 

IV. 

set 

A 

A 

A 

A 

A 

A 

A 

eg 

fH 

|v. 

to 

fv* 

eg 

rv* 

f*i 

o 

CO 

eg 

eg 

a 

eg 

CO 

to 

iH 

eg 

<t  00  eg 

00  CO  *-i 

o  tn  00 

OV 

eg 

tv  OV  to 

eg  VD  oo 

0i 

ov  CO  CO 

VO  "J"  r-l 

vO  4  ov  ov 
•H  O  VO  00 

CO 

eg 

IV.  o  N 

CO  VD  Ov 

ov  CO  CO 

r-l  CM  St 

O 

o 

vD  vo  eg  to 

^  rg  m 

& 

A  A  A 

AAA 

.... 

A 

A 

A  A  A  A 

AAA 

> 

iH  CO 

<<1  VO  Ov 

1-4  M  3 

VD 

o 

H  ov  ov  O 

^g  H  eg 

OV  Ov 

CO  ^s  O 
tH  cv| 

»-i  m  vo  p) 
f-i 

tv 

tv. 

Ov  cjv  eg  eg 
eg  gt 

gj*  VO  O 
r-g 

00  00 

rH  I  t-l 
«»  I 

^  <1- 

VO  VO 


O  O 

M  C4 

CO  •  CO 
««  I  •k 

CNJ 

Ov  OV 


rv 

|v. 

tn 

rg 

l-t 

eg 

fH 

CO  1 

•  CO 

o 

e: 

A  1 

1  A 

A 

A 

VO 

VD 

Ov 

VD 

OV 

rv 

rH 

eg  eg 
CO  CO 
00  I  00 

A  I  A 

00  00 
o  o 


o  < 

W  \ 
(d  CO 
^  00 
00 

O  •*-< 

O  M  M 

<g 

(U 


C3 

Cd 

4J  -U  f-H 

a  a  Cd 


T  a  Eh 

CO  O 
tI  CO 

&  y 


<U  ^ 

u  o 
o  a 
a  p 

>  «  n 

£  CO  r-l 
m  4>  Id 
C  r-t  4J 

h  S  «  O 
®  6  tJ  H 

4J  U  U  01 


01  ®  4J 

,0 

§•5 

> 


o  ® 

r-t 

®  0 
<H  60 
>4-(  M 
O  3  -H 
■H  «  a 

•a 

n 


V  a: 

"d  M 

h 

«  r-l 

U  a) 

4J 

n  o 

<  ^ 


I 


o  o  o  o  o 

o  o  o  o  o 

o  00  VO  \o  o 

M  «  M  M 

Sf  VO  O  C>4  <!■ 

rH  ^  fx 

r-<  H  lA 


o  2 

o  o 

C'J  I  I  I  ^4 

«  I  I  I 

r>* 

csl  C'i 

es 


o  o 
o  o 

VO  vO 


r4 

CV4 


O  O 
O  O 
VO  t  vO 
A  I  A 
VO  VO 

m  m 


o  o 
o  o 
O  I  o 

A  I  A 

tn  m 

a 


o  o  o  o 
o  o  o  o 

00  VO  H 

A  A  A  M 

ov  O  in 

cvj  cn  o  vo 


o  o 

o  o 

<t  I  • 

All  A 

00  00 

m  lo 

evj  CM 


o  o  o 
o  o  o 

iri  ov  ^ 

AAA 

00  r-4  O 
rn  o  CO 


o  o 
o  o 

n*  I  rs 

A  I  A 

cn  CO 
in  *0 


O  O  O 
O  O  O 

00  o 

AAA 

i-i  o  eg 
m  o\  <f 


o  o 
o  o 
00  I  oc 

A  1  A 

00  00 
VO  vO 


.265 

.602 

.650 

h*v 

rH 

•a- 

in 

ON 

CO 

CM 

CO 

A 

A 

A 

OO 

vo 

eg 

CO 

eg 

J  Ov  ^  O  0^ 

iH  Ov  VO 

on  cvj  <0  v-f  -< 

A  A  A  A  A 

^  00  CM  ON 

CO  ON  ir>  vo 


ds  dj 

I  I  I 

A  I  I  I  A 

ON  ON 

a _ a 


in  m 
in  m 

ON  I  ON 

A  I  A 

m  m 
m  m 


eg  w 
00  00 
^  I 

At  A 

3  3 


00  CO  o  »-* 
ON  CO  ON 
f-4  CS  N 

A  A  A  A 

r*.  eg  VO 
^  H  eg  oo 


vo  vo 

CO  CO 

vo  I  I  vo 

All  A 


ON 

O  ON  o 
CO  CO 

AAA 

H  00 
r*H  in  y 


o  o 

ON  CJN 

VO  I  vo 

A  I  A 

OO  00 


H  CO  ON 
CM  Ov  i-l 

A 

O 

m  'T  O' 


p-j  pj 

•O'  <1; 


V  (4 
fl)  "o  u 

a  o  o 
1-1  (0  H 
U  M 
>  (d  «) 
w  > 

ass 


A 

3  ^ 

a  O  60  <4 

g  ii.  60  w 

^^(40 
60  H  H 
b  1-1  « 

Q  0) 

-I.  rH  • 

,d  (4  W 
60P4  h 

•r* 

4) 


O 

«  4)  J 

O  O  I-I 
C  4J  4J  (4 

«4  fi  B  W 

a  s  8  a 

O  4)  B  n 

U  U  U 

B  O  O  O 

4)  O  (4  (4 

B  X  ^ 


V  U  C4 

41  4)  4-1 

.!<:  o 

§  S'" 

ft 


(0 

<u  ft 
O  «  rj 
r-i  »  C^ 
4J 

*0  ^  4J  O 
CS  H  H  H 
eg  o  o 


-  101  - 


F.  COST/BENEFIT  ANALYSIS 


Rationale;  If  one  had  an  estimate  of  the  costs  required  to 
purchase  a  new  option,  and  he  had  a  dollar  measure  of  the  benefits 
they  were  expected  to  provide,  a  simple  choice  mechanism  would  be  to 
purchase  the  option  if  the  benefits  were  any  amount  greater  than  the 
costs.  Notice  that  the  extent  of  the  bargain  —  the  excess  of 
benefits  over  costs  —  does  not  enter  into  the  choice.  This 
mechanism  for  choosing  is  sufficient  for  the  decision  at  hand,  one 
which  is  limited  to  a  choice  between  two  competing  systems;  ARTS 
and  manual.  It  is  not  a  sufficient  mechanism,  however,  for  deciding 
whether  some  other  expenditure  might  not  be  more  rational,  for  the 
FAA.  It  cannot,  tor  example,  be  used  to  decide  whether  it  is 
better  to  invest  in  the  construction  of  improved  runways,  or  in 
new  ways  to  maintain  navigational  facilities,  etc.  Nor  can  this 
rationale  be  used  to  decide  whether  the  Nation  were  better  off 
in  investing  in  welfare  reform,  better  schools,  cancer  research, 
or  farm  subsidies.  These  decisions  are  left  to  others. 

In  the  preceding  sections  the  difficulties  in  ascribing  dollar 
benefits  to  a  new  system  designed  to  perform  a  complex  job  were  dis¬ 
cussed.  The  usual,  but  important,  problems  of  attributing  dollars  to 
such  items  as  1)  lower  variability  in  controller  performance  or,  2)  the 
identification  of  emergency  situations  were  cited  as  examples.  Moreover, 
the  problems  associated  with  the  double  counting  of  benefits  was,  likewise, 
discussed.  There  were  some  80  different  indices  used  in  this  study,  all 
intending  to  measure  a  different  dimension  of  the  controller's  job,  but 
many  were  probably  redundant  reflections  of  the  very  same  facet  of  this 
Job.  However,  this  latter  difficulty  is  frequently  regarded  as  more  of 
a  loophole,  than  a  problem.  If,  for  example,  there  were  an  infinite 
number  of  beneficial  attributes  associated  with  a  system,  then  it  would 
be  possible  to  justify  the  most  expensive  of  systems,  merely  by  assigning 
the  modest  estimate  of  one  dollar  per  attribute.  Is  it  any  wonder  then 
why  many  research  companies  being  paid  to  perform  cost /benefit  analyses 
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st&y  up  late  at  night  thinking  up  more  attributes,  or  nev  naaes  for  old 
ones?  There  is,  of  course,  a  need  to  close  this  loophole  by  iaq>osiiig  a 
strict  discipline  upon  the  number  of  attributes  for  uhich  dollar  benefits 
can  be  claimed:  they  must  represent  an  independent  facet  of  some  essential 
dimension  of  a  system's  performance. 

This  restriction  is  met  for  the  present  stu^  by  limiting  the  number 
of  indices  for  which  dollar  benefits  are  claimed  to  a  single  one  — 
index  54.  This  index  measures  the  time  that  certificated  controllers 
were  in  conflict  with  aircraft  of  the  iFRx  VFR  variety  while  working 
identical  traffic  samples  with  both  the  ARTS  and  manual  systems.  This 
time  in  conflict  is  then  translated  into  the  probability  of  a  midair 
collision.  The  avoidance  of  such  collisions  is,  undoubtedly,  an  important 
function  of  the  controller's  job. 

In  general,  the  method  for  determining  whether  a  specific  terminal 
location  qualifies  for  an  ARTS  III  installation  will  be  to  compare  the 
costs  of  Installing  and  maintaining  this  system  with  the  benefits  from 
the  expected  reduction  in  numbers  of  midair  collisions  at  this  location. 

A  cost  CO  benefit  ratio  which  is  less  than,  or  equal  to,  unity,  qualifies 
the  terminal. 

Summary  of  Method  for  Determining  the  C/B  Ratios 

The  incremental  costs  needed  to  purchase  ARTS  III  equipment  at  all 
29  sites  is  estimated  belcw  to  be  $33.4  million.  Suppose  a  midair 
collision  to  have  a  dollar  (dis)  benefit  of  $22.2  million.  Then 
33.4/22.2,  or  1.5  midairs  would  have  to  be  avoided  for  the  entire  useful 


s 
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life  of  tbe  equ^eent  ia  order  to  garraat  die  purchase  of  ABTS  III.  The 
useful  life  of  ABTS  III'  is,  pr<^ably,  close  to  20  jears,  but  since  all 
new  ieproveaents  in  technology  are  usually  vulnerable  to  being  aade 

obsolete  prenaturely  by  neser  iuproveneats  is  technology,'the  aore 

£ 

conservative  estimte  of  ten  years  was-  used  in  the  study.  For  the  sake, 
of  aritbaetic,  assuae  that  there  are  30‘  installation  sit^,  so  that 
1.5/30,  or  0.03  aidairs  anst,  on  the  average,  ^  be  avoided  at  each 
terminal  location  included  in  the  third  lot  buy  for  the  entire  period 
of  ten  years  in  order  to  'justify  the  purchase  of  ABTS  III- 

Otber  estimates  for  costs  ($C)  and,  (dis)beaefits  of  midair  colli- 

1 

X  !  - 

sions  ($B)  define  a  new  ratio  that  est^lisbss  the  reduction  in  midair, 
collisions  that  are  necessary  to  justify  the  ptnrcbase  of  ARTS  at  each 
terminal  location.  The  analysis  ^ich  follows  wp.!  attenq)t  to  estimate 
this  ratio  by:  1)  determining  tbe  ten  year  system  costs  that  are. 
envisioned  for  tbe  ARTS  and  tbe  manxial  programs;  the  difference  in  these 
system  costs  the  numerator  of  the  ratio,  2)'  estimating  the  value  or 
dollar  benefit  resulting  from  avoiding 'the  occurrence  of  a  midair  colli¬ 
sion.  This  .benefit  was  estiaated,  at  first,  as  a  parameter  value  that 

( 

depended  upon  the  decision  maker’s  subjective  valuation  of  the  expected 
nxmiber  of  fatalities  per  midair  accident,  and  his  estimate  of  the  dollar 
worth  of  each  human's  life.  The  nimiber  of  midair  collisions  expected  at 
each  terminal  location' was  derived  from  the  historical  record  of 
accidents  occurring  in  our  Nation's  airspace.  The  reduced  numbers  of 
accidents  expected  at  each  location  were  based  upon  the  results  of  the 

I 


sillatrd  ^xpezimemti  iaiex  54.  Ifae  mmmtrf  gra^  lisaxe  10  129) 

is  pge«eated  ss  the  xesDltiag  ane^rtleal  tool  for  deciJii^  betoece  tie 
4ns  and  — <  tyetam*  of  sir  txafrie  coctrol.  Tigace  10  iadicstes  the 
seine  that  aost  he  placed  oa  a  aidafr  collision  ia  aniex  to  josti^  the 
porchase  of  AXIS  HI  for  a  tezaiasl  harisg  aaj  giscc  lesel  of  traffic 
activity.  This  is  a  coavenlfiDt  «y  to  avoid  heviag  to  aaSce  sobjective 
judgeents  ahicii  are  better  left  to  others.  Ihe  decision  aalcer  is  required 
to  set  a  dollar  value  on  the  cost  of  svertlps  an  aeddest  that  he  rhiaks 
is  justified. 

ficaever,  a  study  dsaliiig  «fith  xeslistic  (daoices  that  natter  cannct 
afford  the  lumry  of  avoiding  key  issoes.  The  sto^  east,  at  least, 
present  a  guideline  for  decision  asking.  For  this  reason,  an  estisats  of 
dollar  costa  per  accident  is  presented  in  Table  24  for  a  variety  of 
aidair  collisions  involving  several  sizes  of  aircraft,  vith  differing 
ntnibers  of  occupants,  earning  various  levels  of  iscoae.  The  decision 
asker  can  then  enter  Figure  10  with  eny  value,  based  upon  bis  sthjective 
estimate  of  the  dollar  benefits  to  be  derived  frca  everting  an  accident. 

A  aatris  of  collision  probabilities  likely  to  occur  in  the  next  decade  is 
shown  in  Table  23.  Or,  he  can  choose  to  enter  Figure  10  with  a  conservative 
estiaate  —  the  least  costly  end  of  the  spectrua  of  values  of  the 
dollar  benefits  for  avertli^  a  "typical"  accident.  This  accident  is 
represented  by  the  weighted  average  of  the  matrix  of  collision  possi¬ 
bilities  shown  in  Table  23.  By  using  the  least  costly  estimate  for  an 
average  accident,  the  decision  maker  would  conclude,  as  the  study  does,  that 
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oi£  IXE  ^^sigacsS:  '^s  jijstfifffffid  sc  oiciy'  ^sasSscl. 

IgcadeoB  aacloofii  isa  ft&e  cMciS  IsS:  hag. 

COST  gSgg^-i^'gg!^ 

•guTTeaecaail  Cacsits 

CsKSt  eaapil^isas  acre  Isased  cq  data  bg  t&ie  latifisal 

Mrssace  System  greRtam  Offfre  of  tiie  ESUl  fee  t&tec  exangle  locatsoss. 
Iff'fflac  office  is  c&atsed  vita  c£ie  seswKssi&ilitj  for  ioglenestiss  e&e 
AUS  HI  ggosram. 

A.  FiyxiS  Costs; 

Sarnarft  Iccisxille  Birnifcai'iaiu 
Basic  AXIS  III  Gcctract  Cost  $S20,0(D0  ^09,( 

Basic  BUS  III  Is-Socase  Cost  66,OCO 
BIC  Expaasum  Contract  Cost 


17/ 


36,400 


66,000 

38,a»0 


^«B,000 

66,000 

54,400 


Fixed  Cost  per  Iccatioo 


Total  Fixes  Costs 
<30  locaticcs) 

Airrmal 

Operating  Costs;  A3TS  III 


$622,400  $613,800 
(S  10)  a  10) 


Hanuai 


$629,400 
^  10  locations) 

$19.0  millioas 

Difference 


Biraieghaa 

Louisville 

Burbank 


5137,120 

i99,2{M) 

125,280 


$  85,480  $  50,640  (&  10  Iccaticos) 

119,200  80,0(K)  (X  10  locations) 

35,920  89,360  (Z  10  locations) 

$2,200,000  (30  locations) 

B.  Total  Operating  Cost  Iccretsent:  (10  years;  30  locations)  =  $22  .C  Ss-.llions 
Total  (A  +  B)  $41.0  nilliom 


17/  The  three  exanple  locations  were  selected  to  represent  the  lew,  aedina 
and  high  points  of  the  spectnsn  of  candidates.  Each  exaaple,  there¬ 
fore,  represents  one-third  of  the  total  nnaber  of  approxinately 
30  locations;  or  10  locations. 


cott’^Uesrs 


see  cirt—i  £sr  tte  artactSffia  of  JODS  HI  ac  29  c»ifiiitr  c»?»Eatt1s 
ia  tke  maat/t  of  $200,000  fsr  cadt  I-oeaar?ff»  amts  a  g<gio<  aif 
10  jeaxa;  a  tosal  of  $S.8  ttilliaa.  Ibis  ft ff ware  £s  baae£  agoc  sIk 
aoKic  j^ezfitnad  hj  Ike  aufig  Cacyaratioa- 

—  $  5.8  fltillifflo 

Saaiass  etc  i  Irliwa'*  Is  tiie  asrwafr  of  8±e  ci<-A'ai rar-l.  caacellatlae 
coats  fas  aot  rsriTHsfag  tiie  option  to  grocAje  tbn  tMzd  lot. 

~  S  1.8  allllgtB 
Itotai  10  ysax  saviags  ~ 

T9tal  lDC.rfaf.at;  (Total  %X9rC^ 

lacxeaent  per  Iterafaal  $111,300 

($33.4  aEillioas/30) 


18/  For  the  detailad  calcolations  and  assoaptions  see  I9ie  HTISE 
Corporation  report,  Sef.  8. 


A  suanecy  cissliae  e£  rae  loerSos  cse£  Co  calcslass  eoe  orarfftts 
aierlSct^le  £0  a  sgccffffc  ISIS  Ill  caad&fzsa  Isca3:££nsi  is  Eelstv. 

Ibe  disccasiga  gmsaa  follsss  is  layed  ea  Sais  Gc^lioe: 


A.  FsiriiTMinft  of  Ufanfeer  or  gficafir  Qgllisiccas  liSalv  to  gegcr  ia  Fff  1S>77 
I&e  tuSsffflaicaEg  rscozti  c£  irwifatf  seciiawffcs  is  a&ngea  is  l^le  13  Ecx 
g&e  Tcazs  1938  tfergraga  137S.  It  is  lixelj  Cost  acci/Seass  sill 

fggirease  &ariag  eae  tact»  dSe<»3e  cse  co  1)  as  iacireasfc:  ia  eocsaAioes, 

2)  a  Isggcr  crass*se£C£2ffial  cjallisiaa  ases  &ne  co  ioczcaseS  oancers  of 
bigpeg  sized  aircraft,  3)  iacgeased  soeeds  fsera  Caa  igfcrodzcfciga  of  asre 
canfeegs  of  Jei  aizcsaft.  Cts^  tbs  first  of  cbese  factcrs,  gacreasga 
cgeratiassss,  sas  ircioded  ia  tbe  stray's  cstinate  of  tas  aanfeer  of 
acci<5ezts  liiely  to  occar  ia  Ff  1977. 

Aa  aaal3rsis  sas  irrade  of  tee  relatiaasaip  densastrated  betseea  ebe 
aaober  of  arcideats  and  traffic  activity  levels.  Since  a  rnicair  collisii^ 
is  a  rare  eveat  it  is  diffiedt  to  disceca  aa  objective  nattera  capable 
of  ideatifyiag  pt^esrial  hazards.  For  tbis  reasc^e,  tbe  data  of  acteei 
midair  collisions  sas  si^leaeated  an  analysis  of  the  nsmber  of 

19/ 

hazardons  near  misses  in  an  iasfepenaeat  sts^  conducted  by  tbe  FA&.  — 
Table  20  depicts  tiie  caabar  of  near  miss  incidents  recorded  at 
various  major  terminal  locations  for  tbe  jiear  1968.  Ihe  traffic 
activity  during  1963  for  these  terminals  is  sbosn  in  tbe  ri^t  band 
coltsm.  Ibis  activity  includes  VFR  traffic  at  all  teser  equipped 


19/  ISear  Midair  Collisixm  Report,  1968;  see  Ref.  9- 
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IH'enai’id  by  FB-90 
3/2/71 


■fcondeTy  airpocts  widiin  tbe  mge  mad  jarisdictioa  of  the  primrjr 
s£zporS:*s  radar.  The  graghical  pzesestatioc  o£  tbese  data  in  Figiixe  8 
reveals  that  a  near  proportional,  statistically  significant,  relation- 
ship  exists  betseen  the  ncmiier  of  near  itcisses  rqpsrted  and  the  record 
of  traffic  activity,  ^liiis  relationship  nas  used  helov  to  estinste  tSie 
nonher  of  near  niss  incidents  that  are  likely  to  occor  at  aiy  given 
temiial,  hot  for  purposes  of  the  discassion  at  hand  this  relationship 
was  first  nsed  to  estinate  the  total  needier  of  nridair  accidents  that 
can  he  expected  to  occur  in  ST  1977.  Ibe  nnaiber  of  "lidair  accidents 
reported  in  1968  nas  38.  Ifae  ratio  of  the  nnnber  of  total  opexatioos 
forecast  for  1977  conpared  to  1958  is  1.64.  Assoming  that  hazardous 
sear  nisses  provide  a  good  prorj  for  actual  accidents  (the  clidie,  ’4diere 
there's  snolce  there's  fire"  vill  have  to  suffice  in  place  of  a  nore 
eluant  eaplanatioe  sdii<di  ssgrs  the  saae  thing),  we  have  a?  logical  basis 
for  expseting  that  the  coaber  of  nidair  accidents  in  1977  will  he 
38  X  1.64  >  62. 

The  identical  aethod,  without  the  explanation,  is  used  by  the 
Sational  Transportation  Safety  Board  to  forecast  their  estinate  of  the 
nuidier  of  accidents  likely  to  occur  in  the  next  decade. 

'forecasts  indicate  a  growth  by  a  factor  of  1.7  in  tbe 
general  aviation  fleet  and  by  a  factor  of  1.5  in  the  air  carrier 
fleet  over  the  next  10  years.  In  the  saae  tiae  period,  tbe  total 
nuaber  of  operations  of  these  fleets  will  be  half  again  aa  large 
aa  they  are  today «  Assuming  tbe  accident  and  fatality  rates  are 
the  sane  for  the  next  10  years  as  th^  were  for  the  last  10,  we 
would  expect  the  raimber  of  such  accidents  to  increase  by  50  percent; 
that  is,  335  accidents  and  792  fatalities  for  tbe  10  year  period."  ^0/ 

MTTEpo^of  proceedings  of  the  HTSB  into  the  Midair  Collision  Problem, 

p.  1;  see  Paf.  10. 


s. 
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For  eMs  study,  tbSi  mabex  of  opsrations  foreca^ed  for  FF  1977 

21/ 

«ere  obtained  frotn  official’  FAA.  estiaates:  — 

FY  1%8  FY  1977  Ratio 

Total  Itineraat  Cperaticms  32.4  53.2  1.64 

(millions) 

B.  Proportioa  of  Midair  Accidents  at  an  Individual  Teminal 

This  section  of  the  study  attests  to  detemine  hou  the  nuidier  of 
aidair  accidents  forecast  for  FT  1977  (a  total  of  62)  will  te  distributed 
aaong  those  teiaical  locations  identified  in  ^be  third  lot  buy  of  ARTS  Ill 
eguipoent.  The  probability  that  one  of  these  locations  will  have  an 
ARTS  III  preventable  accident  is  represented  hy  the  product  of  three 
separate  probabilities: 

1)  The  probability  that  the  accident  will  occur  in  a  texminal 

area. 

2)  P2  =  The  probability  that  the  accident  will  be  of  the  "unknown" 
vs.  *Tcnown"  (IFR  x  VFR)  variety. 

3)  P3  =  The  probability  that  a  given  third  lot  buy  location  will 
be  one  of  the  terminals  at  which  the  accident  will  occur. 

The  calculations  of  these  separate  probabilities  are  shown  below. 

They  are  based  on  the  ''Near  Miss  Collision  Report  of  1968."  The  data 
from  this  report  used  in  this  study  are  reproduced  as  Table  21. 

The  probability  of  a  midair  collision  occurring  in  the  terminal 
area  was  calculated  as  the  ratio  of  the  number  of  near  misses  reported 


21/  "Aviation  Forecasts,  Fiscal  Years  1972-1983,"  Office  of  Aviation 
Economics,  FAA;  September  1971. 


Table  20 


Record  of  Hazardous  liear*iCisses  in  Terminal  Areas,  1968 


I 

I 


Near'Kiss 


Location 

Incidents 

1.  Los  Angeles 

74 

2.  Heir  York 

53 

3.  San  Francisco 

38 

4.  Vasbingtcn 

24 

5 .  Philadelphia 

15 

6.  San  Diego 

19 

7.  Chicago 

21 

8.  Phoenix 

12 

9.  Kansas  City 

11 

10.  -Denver 

17 

11 .  Colunbus 

6 

12.  Dallas 

10 

p.  Seattle 

11 

14.  Detroit 

7 

15 .  OklahcKua  City 

9 

16.  Atlanta 

6 

17 .  Indianapolis 

5 

18.  Minneapolis 

9 

19 .  Norfolk 

6 

20.  Cleveland 

7 

21.  Miami 

22 .  Tampa 

7 

23 .  Dayton 

6 

24.  Honolulu 

8 

25 .  Houston 

7 

26 .  Sacremento 

8 

27 .  St .  Louis 

8 

28.  Boston 

6 

29 .  Buffalo 

6 

30.  Las  Vegas 

7 

31.  Memphis 

6 

32 .  Pittsburgh 

3 

33.  Portland,  Ore. 

3 

34.  San  Antonio 

4 

35 .  Louisville 

3 

36.  New  Orleans 

3 

37 .  Cincinnati 

2 

38.  El  Paso 

2 

39.  San  Juan 

2 

40.  Nashville 

2 

total  near  misses 

496 

CT  1968 

Traffic  Activity  (ICr ) 

IFR;  VFR 
87.8 
151.1 

131.4 

45.6 
52.5 

76.7 
109.0 
105.6 

44.7 
43.0 
42.0 

61.4 
47.0 

69.8 

37.9 

59.5 

19.7 

54.7 

41.4 
36.0 

124.5 

46.7 

31.9 

30.4 

41.4 

47.2 

39.2 

46.5 

20.7 

35.5 

21.2 

36.8 

22.3 

67.9 

26.7 

36.6 

22.9 

15.8 

29.3 
17.1 

Total  Primary  and  Secondary  Airports 


SOORCE;  Near  Midair  Collision  Report  of  1968,  FAA,  July  1969. 

Traffic  ActivifjTj  Office  of  Aviation  Economics,  EC-200 
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in  the  terminal  area  (719;  from  Table  21)  divided  by  tbe  total  number 
of  hazardous  incidents  recorded  (1,128  p.  lx  of  summary  to  report). 


»  -  719 
1128 


0.64 


Ibe  probability  that  a  collision  will  be  between  an  IFR  and  a  VFR 
aircraft  was  calculated  as  the  ratio  of  the  number  of  incidents  of  this 
type  in  the  terminal  area  (395,  as  shown  in  Table  21)  divided  by  the 
total  numter  of  terminal  incidents. 


P2* 


(217  +  54  -f  124) 
719 


0.55 


The  inclusion  of  this  single  category  of  collision,  IFR  x  VFR,  is 
necessary  in  order  to  make  the  inferences  drawn  from  the  real  world 
conform  to  the  experimental  findings :  Index  54  is  a  measure  of  the  time 
in  conflict  for  the  specific  interaction  of  IFR  x  VBR  aircraft. 

The  probability  that  an  accident  will  occur  at  an  individual  terminal 
included  in  the  third  lot  buy  was  postulated  to  depend  on  the  traffic 
activity  at  this  terminal.  Table  20  Indlcater  the  terminal  locations  and 
traffic  activity  for  496  incidents  recorded  out  of  the  total  of  719.  A 
graphical  presentation  of  the  postulated  relationship  between  the  number 
of  hazardous  near  misses  reported  and  traffic  activity  is  shown  in 
Figure  8.  (The  data  are  keyed  to  Table  20).  The  New  York  and  Atlanta 

f 

facilities  were  excluded  from  the  data  used  in  determining  the  estimated 
relationship  because  these  facilities  had  ARTS  equipment  in  place  during 
1968.  In  the  case  of  Atlanta,  this  equipment  was  fully  operational  in 
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Vabie  21-  ‘ 


TERMINAL  HAZARDOUS  INCIDENTS  (LOCATION) 
(NETWORK  ANALYSIS) 

ALTITUDE 


TERMINAL 


1st  Quarter 
2nd  Quarter 


3rd  Quarter 

200 

4th  Quartw 

142 

Total 

719 

.  ’INVOLVEMENT 

1®Q  2®Q  3°Q  4°Q 

2 

3 

3 

1 

9 

AC/GA 

68 

61 

86 

56  271 

AC/MIL 

12 

15 

10 

10 

47 

GA/GA 

57 

40 

57 

36 

190 

GA/MIL 

55 

46 

37 

36 

174 

MIL/MIL  ,  , 

11 

7 

7 

3 

28 

TURBO/TURBO 

14 

10 

13 

5 

42 

TURBO/PISTON 

109. 103  104 

89  405 

TURBO/HELO 

3 

1 

2 

1 

7 

PISTON/PISTON 

75 

55 

72 

46  248 

PISTON/HELO 

4 

3 

9 

1 

HELO/HELO 

0 

0 

0 

0 

GROUND  UP  BEYOND  5  Ml. 

1st  Quarter  ■ 

66 

2nd  Quarter 

81 

3rd  Quarter 

90 

4th  Quarter 

61 

298 

2,000  FT.  AGL  AND  ABOVE 
WITHIN  5  MILES 

Ist  Quarter 

33 

2nd  Quarter 

15 

3rd  Quarter 

18 

4th  Quarter 

17 

83 

Ist  Quarter 

106 

2nd  Quarter 

76 

3rd  Quarter 

92 

4th  Quarter 

64 

338 

TYPE  OF  FLIGHT 


IFR/IFR 


1°Q  2“Q  3°Q  4°Q 


43  57  66  51  217 


I^Q  2‘’Q  '3°Q4‘’Q 


21  19  23  7  70 


IFR/IFR 


1“Q  2“Q  3“Q  4°Q 


1 


IFR/VFR 


1®Q  2®Q  3®Q  4“Q 


21  10  10' 13  54 


VFR/VFR 


10  5  7  .4  26 


1®Q  2®Q  3°Q  4°Q 


1  1  1 


IFR/VFR 


1°Q  2®Q  3VQ  4®Q 


35  28  32  29  124 


VFR/VFR 


1®Q  2®Q  3“Q'4' 


I  69  47  59  33  208 


SOURCE;  Reproduced  from  "Near  Midair  Collision  Report  of  1963," 
Appendix  C-1,  p.  119, 


1968.  The  Lqs  Angeles  terminal  was  also  excluded  as  a  special  case  or 
"outlier."  :The  number  of  near  misses  recorded  for  the  Los  Angeles 
terminal  area  was  more  than  five  standard  deviations  removed  from  the 
average- value  expected  for  a  terminal  with  its  level  of  activity.  (In 
the  case  of  los  Angeles  the  level  of  .;raffic  activity  appropriate  to  the 
estimation  of  near  misses  is,  obviously,  understated  in  Table  20.). 

The  number  of  near  misses  reported  in  the  terminal  area  was  found  to 
be  very  nearly  directly  proportional  to  traffic  activity  as  shown  in 
Figure  8.  The  estimating  equation  shown  is  statistically  significant 
and  accounts  for  approximately  50  percent  of  the  variance  in  the  data 
for  near  misses  in  the  terminal  area.  This  represents  an  appreciable 
explaining  capability  when  one  realizes  how  varied  are  the  reasons  for 
a  midair  collision.  Many  factors  influence  and  contribute  to  the 
generation  of  a  single  datum  point  for  an  accident  or  near  miss,  and  yet, 
we  can  explain  one  half  of  the  variation  in  these  data  with  a  single 
explanatory  variable;  traffic  activity.  Besides,  this  variable  is  the 
sole  criterion  being  used  for  the  Installation  of  ARTS  equipment.  If, 
for  example,  weather  conditions  or  other  variables  were  included  as 
significant  explainers  of  near  misses,  then  it  would  be  imperative  to 
include  these  other  variables  as  part  of  the  conditions  required  for  a 
terminal  to  qualify  for  ARTS  III.  Since  there  is  no  stated  intention  to 
impose  an  array  of  installation  criteria  for  ARTS  III  —  traffic  activity 
is  sufficient  —  and,  since  there  is  no  evidence  that  the  incremental 
impact  of  increasing  traffic  levels  on  near  miss  incidents  will  be 


100  ,  NO.  HAZARDOUS  NEAR  MISSES  IN  TERMINAL 
FIGURE  goL  ,  VS  TRAFFIC  ACTIVITY 
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affected  by  the  inclusion  of  other,  unspecified,  variables,  it  is 
appropriate  to  use  the  equation  shewn  in  Figure  8,  as  the  basis  for 
estiniating  the  number  of  near  taisses  at  a  t^^minal.  — ^ 

Figure  9  was  redrawn  to  include  just  those  terminals  included  in 
the  third  lot  buy.  An  adjustment  was  also  made  to  make  the  relationship 
between  near  misses  and  traffic  activity  directly  proportional.  (The 
exponent  of  0.98  was  changed  to  l.O.  The  better  way  to  explain  this 
changa  is  to  state  that  the  data  do  not  refute  the  hypothesis  that  the 
relationship  is  proportional).  The  number  of  near  misses,  Y,  at  a 


terminal  location  with  traffic  activity,  X  (in  millions),  is  given  by: 

Y  «  17.8275X  ( 


The  percent  of  variation  in  the  data  that  is  explained  by  the  single 
variable,  traffic  activity,  for  these  fewer  numbers  of  terminals,  is 
65  percent  (statistically  significant).  A  comparison  ot  the  actual  near 
misses  recorded  at  ARTS  III  candidate  locations  with  the  numbers  estimated 
by  Equation  (1)  Is  shown  in  Table  22. 

The  probability  that  a  near  miss  incident  will  occur  at  an  individual 


terminal  is  given  by: 


Near  Misses  at  Individual  Terminal 
Near  Misses  at  All  Terminals 


17.8275X 


(X  =  traffic  activity  in  millions) 


0.25X 


22/  There  is  no  need  for  this  exegesis  if  we  go  along  with  the  NTSP 
and  ''assume"  that  accidents  will  be  proportional  to  traffic 
activity;  see  Footnote  20.  The  study  preferred  to  conduct  an 
independent  analysis  in  order  to  confirm  the  relationship. 


SSCICE:  Ibid.  8. 
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23/  As  *^istorical  Deveix^aent  of  the  *See  aad  5e  Seen’  Csra^pt”  is 
provided  bj  the  STSB  as  Appendix  3  to  their  saasazy  of  the  pro- 
ceedip^  of  the  Hidair  Collision  Problem;  ibid.  Sef .  10. 
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3)  IBs  final 


fa  si:a»3i«rS:  <sf  Sis 


S-ngi-  ffTTg* 


pnc£s^ili£^  of  a  aadair  collisica  is  pscgagfcicsal  Co  Ccs  Sine  in  c<o~ 
flict  is  prgyjdei  cy  Si:a  follccrisg  seasoniES: 

IBs  cagitenticG  is  oartainlj  tena  foo:  a  ccnflicCis^  airspace  teat  is 
de  inter^cSioa  of  rec.Cangalar  parallelpipsd  Toltnes  Snac  carely  coi^ein 
Cro  airczafC.  Ifs  lesser  an  aircraft  is  in  snea  ti;g^t  quarters,  tie 


24/  See  2ef-  ii;  vbl.  I,  p-  47-  Appendix  C-3,  diicdi  descriies  tie  vork 
of  Granan  and  Orr  is  also  of  interest  in  that  it  aretes  the  nore 
general  proposition  that  the  probability  of  a  nldair  collisicm  is 
proportional  to  tie  tine  spent  in  the  tezninal  area. 

25/  See  Bef-  32- 
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ferae  eis  cggffgias  ctf  &&  gasaSlglpigs  taoese  grfeertasS  jesfe  s  littttia 
fem  aggigerg  &c:  2|  pilrtt  rffiasfeSaa  Sr&ns,  2)  cacfrra'glgg  ceac&fso  fesme, 

^  «£rt3  cseSEeSocs,  4]^  aaecrasy  of  Css  fg?sCj.mig?g:s  cd^sallfrs 
fexafcic;  tasars,  alfisaiefeffirs,  <e£c-,  ard  5>  all  cfe&er  sysfeem  esr&irs 
aofl  -wErfaeSscs  ia  da&a-  if  ws  eifeesfied  dfoersifflcs  ©f  «H=!»r>  ai-r-w 

S^SSS  feiSS  f CST  prScSSSly  ffr^yg^  xSeS^Q  g<r<«ninrTiTa?j»gt«!  Sii.  of 

fere  eioarced  sacsxes  ef  ertoff,  xae  wonid  a  osjsfiictiEg 

zsrs^ssB  as  fesaf  aSrspac?  £a  ‘saics  prc&^iifCy  of  a  m<ri^j=>^y  eollfsicQ 
■3K£S  psoporfeissssl  feo  efe  feme  is  cassiflics:-  £s  a  of  feet,  srea 

a  aafiazCizaa  sa  a  araf  eafEcaac.  for  Ij^gai 

ssaaSasSs  for  s^aratfag  aircraft-  la  oSiar  sordls,  if  3^al  s^acatioa 
stzsdsrds  ware  establisssal  ia  Ciiis  mssrer,  Eqoatioa  (4)  lacaM  ise  troa, 
cy  fefiaitioe-  Hnseser,  if,  is  aot  aecessary  to  offer  fele  record  of 
era  oearirgs  in  ^aaica  txe^  standards  i»ere  esfeablisad  ia  order  to 
support  toe  stop's  corteatioa.  lEsre  zsed  not  nare  beea  a  coascioas 
effort  to  define  airsggce  separatioa  standards  accordiig  to  thp-  abere 
Ascription.  If  tAre  is  a  strong  belief  that  a  raticmal  retbod  of 
alloslig  for  saccessive  iacreaeats  of  airspace  to  accoanodate  each 
fadejsndsnt  soarce  of  error  ia  tas  systea  of  air  traffic  central  vas, 
indeed,  used  ia  Atiaipg  these  standards,  then  the  sane  degree  of 
heHitf  supports  the  contention  that  Equation  (4)  is  a  valid 
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of  tfTn^  zel2££cc^i9  ma&Blr  collisioss  ssS  time 


26/ 

33  ccaflict.  — 


T?Ta  gssspMc^ 


■  of  54,  issclts^sd  33  a  pze^oos 


secticsa  e£  tt£e  report,  is  repeated  Bel®*;  Figcre  5&-  iSe  data  points 
Qssica  are  g^Mafta  r^vreseat  ctjserratiocs  for  th&  first  fealf  Iwor  (f}, 
frpe  seccod  oocr  (s)  ard  tie  coimired  isoar  (C>  of  tie  eiperitnent. 

In  effect,  tie  ralie  slaaaa  for  tie  fell  isocsr  hss  tafice  tie  «ei^t  as 
isalf  isocr  -raices,  and  is  tie  prircipal  firsdir^  of  Site  stcdyr  tie 
aver;^e  ia  conflict  for  all  tears  and  all  sarple  tjpss  esolqyed 
in  frr.^-  experiment  was  149  seconds  per  sinalation  isotsr  for  tie  sanual 
systea;  90  seconds  for  tie  522S  system.  Bqoation  (4)  —  sidair  accidents 
are  proportional  to  time  in  conflict  —  aad  Equation  (1}  —  Esidair 
accidents  as  measured  By  tbe  ngiaer  of  near  aisses  are  proportional  to 
tie  isral  of  traffic  activity  —  is^ly  Equatica  (5>.  Ibe  tise  £a 
conflict  is  proportional  to  the  level  of  traffic  activity. 

T  =  cX  (5) 

Figure  5A  is  s  graphical  presentation  of  this  equation.  Ihe  pro¬ 
portional  lines  were  drasn  through  t;Jie  origin  (when  there  is  no  traffic, 
there  is  no  tise  in  conflict)  and  a  point  which  represents  tie  comibined 


26/  This  argument  was  suggested  by  A.  H.  Schainblatt  of  the  University 
of  California  at  Santa  Barbara  who,  in  the  role  of  a  general  con¬ 
sultant,  was  kind  enough  to  read  the  earlier  drafted  versions  of 
this  study.  A  description  of  an  attempt  to  devise  separation 
standards  by  using  rational  methods  consistent  with  his  argument 
is  provided  by  the  work  of  Holt  and  Hamer;  see  Ref.  11. 


-  J3  6  - 


FIGURES  A 


TIME  iH  CONFUCT-IFR/WR'MIHUTES  PER  SIMULATIOH  HOUR 

VS 

IFR  +  VFR  iOllLY  RATES  OF  TRAFFIC 

IMDEX  #54  ■ 

3  -  CERTinCKD  COHTROLLERS 

^  MANUAL 


TIME  m 
tmmzi 
MINUTES 
PER  HOUR 


65  70  75  80  85 

HOURLY  RATE  OF  AIRCRAFT:  IFR  PLUS  VFR 


ARRIVALS  AND  DEPARTURES 


-  127  - 


hour  value  for  index  54.  The  slope  of  the  line  .foi:  the  time  (in  seconds) 
spent  in  conflict  is,  therefore:  149  for  manual;  90  for  ARTS.  This 
results  in  a  constant  ratio  of  90/149  =  0.6  to  reflect  the  reduced  time 
spent  in  conflict  for  ARTS  at  all  activity  levels.  From  Equation  (4) 
we  expect  that  the  number  of  midair  collisions  will  have  the  same  ratio. 
The  percent  reduction  in  midair  collisions  resulting  from  the  use  of 
ARTS  is,  therefore,  estimated  as: 

R  =  (1.0  -  0.6)  =  0.4^  or  40  percent 

D.  $  Benefit  for  Avoiding  an  Accident 

1)  In  this  section  of  the  study  we  permit  this  dollar  benefit  to 
be  a  parameter;  to  assume  whatever  value  is  required  to  justify  the 
installation  of  ARTS  III  equipment  at  alternative  levels  of  traffic 
activity . 

This  value  is  obtained  by  solving  the  equation; 


(A) 

’  (B)  • 

(C) 

•  (D) 

=  (E) 

62 

0.0088X 

0.4 

$B 

$111,300 

(7) 

The  value  for  E  =  $111,300  was  determined  in  the  preceding  section  as 
the  increment  in  annual  costs  required  to  install  and  operate  an  ARTS  III 
system  for  10  years  at  a  typical  third  lot  location.  An  appropriate 
point  in  time  for  canparing  the  added  costs  required  to  achieve  given 
benefits  is  halfway  through  the  decade  of  intended  use;  i.e.,  FY  1977. 

Figure  10  is  a  graphical  presentation  of  Equation  (7).  It  indicates 
chat  at  the  activity  level  forecast  for  the  Syracuse  terminal  area  in 
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1977,  the  dollar  benefits  from  avoiding  a  midair  collision  must  be  in  excess 
of  $3.  77  million.  All  other  ARTS  III  terminal  areas  have  traffic  levels 
•which  are  forecasted  to  be  higher  than  Syracuse  and  would,  therefore, 
require  lower  dollar  benefits,  as  sho'wn  in  Figure  10. 

In  order  to  complete  our  analyses  of  costs  and  benefits  to  determine 
whether  an  ARTS  III  installation  is  justified  at  a  given  terminal  location, 
it  remains  for  us  to  estimate  the  expected  value  of  avoiding  a  midair 
collision.  On  the  basis  of  our  analysis,  we  will  use  the  decision  rule 
that  an  installation  of  ARTS  III  is  justified  at  Syracuse,  andj  therefore, 
at  all  other  locations  in  the  third  lot  buy,  if  this  value  exceeds  $3.  77  million. 

2)  Determination  of  the  Value  of  Preventing  a  Midair  Collision 

We  have  attempted  previously  to  advise  the  reader  of  the  sub¬ 
jective  judgments  that  are  required  in  even  the  most  quantifiable  and 
objective  portions  of  this  study.  It  is  now  necessary  to  advise  him  that 
this  next  section  of  analysis,  the  dollar  value  to  be  attributed  to  averting 
a  midair  collision  is  inherently  more  subjective.  And  it  should  be.  The 
decision  maker  cannot  avoid  facing  this  decision  head-on  although  quanti¬ 
tative  analyses  can  identify  the  subjective  elements  for  him,  narrow  their 
range,  and  indicate  the  sensitivity  of  the  decision  to  these  elements.  But, 
there  are  limits  to  the  uses  to  wMch  numbers  can  be  put,  and  these  must 
be  understood  as  well.  For  example,  it  may  be  a  delusion  to  attempt  to 
estimate  the  value  of  an  average  accident  when  these  values  are  not 
Jinear,  but  are  a  series  of  discrete  constraints  defined  by  the  whimsical 
moods  of  society  (’']  it  were  only  possible  to  define  society).  This  means 
that  while  it  is  possible  to  estimate  the  costs  of  this  accident  or  any 
number  that  society  will  tolerate,  these  estimates  have  meaning  only  if 
the  "breaking  point"  has  not  been  reached. 

Perhaps,  the  public  will  tolerate  one  more  midair  collision 
between  airliners,  that  is,  if  a  jumbo  jet  is  not  involved.  But,  it  is 
likely  that  two  midair  collisions  would  result  in  a  public  outcry  to  revamp 
the  Nation's  air  traffic  control  system.  Does  it  make  sense,  then,  to 


DOLLAR  BENEFITS  FROM  AVOiDINO  MIDAIR  COLLISION 
REQUIRED  TO  JUSTIFY  ARTS  III  INSTALLATION  AT  TERMINALS 
WITH  GIVEN  TRAFFIC  ACTIVITY 


TRAFFIC  ACTIVITY,  FY  77  FORECAST 
(IFR,  VFR;  Primary  and  Secondary  Airports) 
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estimate  that  one  accident  will  cost,  say,  $10  million  and  that  two  will, 
therefore,  cost  $20  million?  It  does,  only  if  the  world  of  accidents 
remains  linear  and  if  the  "breaking  point"  has  not  been  reached.  However, 
consideration  of-where  systems  break  down  ma'y,  indeed,  be  more  appro¬ 
priate  to, decision  making,  even  though  it  is  often  impossible  to  define  this 
breaking  point  numerically.  Pleas  by  governmental  agencies  to  the  effect, 
for  example,  that  our  "American  way  of  life  is  threatened"  or  "the  edu¬ 
cation  of  children  will  be  compromised"  unless  the  public  makes  a  given 
investment  may  be  realistic  and  highly  relevant  comments,  even  if  they 
are  devoid  of  numerical  content.  The  practical  effect  of  invoking  unquanti¬ 
fied  statements  concerning  where  systems  break  down  is  that  they  may  pro¬ 
vide  no  guidelines  whateveri  All  governmental  agencies  could  plead  that 
a  single  unsafe  incident  in  their  jurisdiction  was  one  too  many  and  the 
decision  maker  would  have  no  rational  method  for  allocating  the  Nation's 
resources  among  many  unquantified  pleas.  However,  it  is  clear  that  the 
assumption  of  a  continuing  linear  relationship  for  the  costs  of  accidents 
that  one  is  required  to  make  in  order  to  provide  numerical  substantiation 
for  the  benefits  claimed  for  ARTS  III  understates  the  advantages  to  be 
derived  from  this  equipment. 

For  example,  suppose  that  there  were,  say, a  97  percent  proba¬ 
bility  that  as  many  as  "Q"  midair  accidents  would  not  occur  if  the  present 
rnanual  system  of  control  were  in  use.  (Q  is  the  level  at  which  society 
withdraws  its  acceptance  of  the  Nation's  air  traffic  control  system. )  A 
central  argunient  of  this  study  is  that  the  use  of  ARTS  equipment  is  likely 
to  increase  this  probability;  say,  to  98  percent.  This  is  an  essential 
piece  of  information  that  should  be  made  known  to  the  decision  maker, 
but  an  analytical  method  which  assumes  that  accidents  may  be  aggregated 
linearly  without  regard  for  the  limiting  Q  value  is  not  able  to  provide 
this  critical  information. 


Since  the  analytical  method  we  intend  to  use  in  the  following 
section  understates  the  advantages  of  ARTS  III — it  is  more  difficult  to 
justify  an  investment  decision  in  a  linear  world  in  which  the  costs  of 

accidents  can  be  aggregated  continuously - this  study  will  assume  that 

such  a  woi'ld  exists,  and  that  neither  the  ARTS  nor  manual  systems  of 
control  will  result  in  numbers  of  midair  collisions  that  exceed  society's 
level  of  acceptance. 


s 


.  r  -  131  -  ■ 

There  are  three  principal  populations  of  aircraft  air  carrier; 

*  t 

general  aviation;  military  —  in  two  flight-rule  categories  —  IFR; 

t  * 

VFR  rr  yielding,  .a. itotal-. of  .uli^  possible  varieties  of  cellisicns 

involving  all  aircraft  populations*  flying  both  sets  of  irules.  Ihc 

•  •  -  - 

matrix  of  possibilities  is  shown  in;  Table  23.. 

The  probabilities  of  each  type;  of  collision  are  shown  in  Table  23 
as  the  product  of  the  relative  proportion  of  aircraft  in  each ‘of  the 
three  categories  of  aircraft  types  and  the  two  kinds  of  flight  rules 
that  were  forecast  to  be  controlled  by  the  Birmingham,  Alabama,  terminal 
in  1977.  Birmingham  was  selected  as  a  representative  facility  in  the  , 

•  s’* 

third  lot  procurement.  .  . 

t  ,  ^ 

Table  24  is  a  compilation  of  the  data  used  to  yalue  an  ac.cident  in 
each  of  the  'population  categories.  Low  and  high  estimates  are  shewn.  The 

•  .  j 

average  value  of  a  typical  midair  collision  (at  a  representative  terminal 

*  1  •  - 

like  Birmingham)  was  estimated  to  be  the  weighted  average  of  the  proba- 

‘  .  •  I 

>  ,  ' 

bilities  and  dollar  values  for  the  nine  possible  types  of  collisions. 

•  St 

A  low  estimate  of  $7.32  million'  and  a  high  of  $120  million  result 
from  this  compilation.  ^  '  '.s' 

The  low  estimate  of  $7.32  million  for  the  average  midair  collision 
expected  in  1977  in  a  terminal  area  such  as  Birmingham,  is  more  than  , 
sufficient  to  justify  the  installation  of  ARTS  III  ,at  ail  third  lot 

I 

i 

locations.  The  margin  of  acceptance  at  Syracuse,  the  low  end  of  the 
activity  scale,  is  about  two  to  one;  $7.32  million  benefitJ  allowed; 

$3.77  millions  expected; 


I 
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-  cas2smzim  cf  ise 

B?  iisica  10  HESSSSSS?  MR,  CSS3ISS. 
S2SL  AVIMim  ASD  Twnry.wagy  -jsiSHIC 
m  GSE  CfMSQSIlE  MlO&ia  OiSlISKM 


Proport  ixEi 
of  IFR 
Traffic  ^ 


(II)  Gk 
(III)  Mil 


AC 

ii 

13 

■IBMi 

.0369  (ixl) 

.0043  (IIzl) 

.C018  (Illxi) 

.4569  (1x2) 

.2896  (11x2) 

.1205  (1112:2) 

(3) 

Mil=  -120 

Scan  =  1 

Notes:  IFR  and  WR  propcrtxons  ware  coi^uted  rroa  txatfic  at  Biraisighaa 
(to  represent  the  airports  in  tbe  third  let  procureasnt)  as  fore 
cast  for  Ff  1977. 

Tbe  cc^uted  ratios  in  this  table  were  mzltiplied  hj  the  lo«  and 
high  (separately)  dollar  values  shown  in  Table  19  to  obtain  the 
following  estimtes  of  the  cost  of  a  nidair  collision: 
l055  estiioate  =  $7,320,000 

High  estinate  =  $120,000,000 

The  formula  for  the  computacion  is; 

3  3 

I  I  iri.i  (V,  +  VO 

i=li=l  J 


Example;  .0069  (12  +  12) 

.0043  (12  +  0.22) 
.0018  (12  +  1.45) 
.4569  (12  +  0.22) 
.2396  (0.22  +  0.22) 
.1205  (1.45  +  0.22) 
.0o32  (12  +  1.45) 
.0401  (1.45  +  0.22) 
.0167  a. 45  +  1.45) 
Weighted  Average  = 

Government  investigation  cost 


.1656 

.0525 

.0242 

5.5833 

.1274 

.2012 

.8500 

.0670 

.0484 

$7.12  million 
.20 


Low  Average  Value  of  Typical  Midair  =  $7.32  million 
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tssa  -SEB  Esspgsgg  Tig:  oosFja  less  eg  a  pfiigais  casEssES^ 


Mr  Carrleg 

Tjsssa  gac&ac 

Cfearfcy 

Talcs  af  sSrerafe 
^lae  «sf  a  life 
Ib&sl  lass 

Military  Aircrafe 
Tsics  of  aircraft 
Sb.  of  <ggTii"yr-fts 
Talre  of  a  life 

Total  loss 

Gararal  Ayiatioo 
Talre  of  aircraft 
Sb,.  af  ocezrsaats 
Tails  of  a  life 
Total  loss 


lets?  Esriasss 


gsgja 


T2 


T3 


3fflS 

SOGS 

1130 

seers 

§s,€aiio,e0O 

$3G«0L/>xC(iI3 

5320-000 

?l2,CGl©,eCi)(!S 

§1^,000,000 

$i,QS©,fflno 

§30,000,000 

1 

£2<K3,Ci!I0 

§300,000 

S15O-G0O 

ii,450,«IffiO 

§135,750,000 

$20,ij®9 

§3,C(!0,C®9 

1 

10 

STfOD.GOO 

5300.000 

$220,(iffi0 

§6,000,000 

SSgxss  tSs  antfss  segfam  (^svatsS  es^  t5e  <?fegVfr:irf!rgg£cin  c£  tse  (Ssllhsr 
&gig££fcs  jifigrETanit^le  co  2si  .^gR-ps  Fa'S  ■f>«TVT?rrsT»'a;>!tfjTO  <rMTyBf!<wr*^  c£  s. 
c£  mrnT’frfg»?fV.<y?qg  SsBsZacs^  aaaaaCfsrSfig?  c£  ^xsoe  grmglimTHjf^m  e» 
ggffi  xRsrfecs  eSsaesES  c£  -g-rta^pgrTr^ieTfr  szsljses  gam  h£  assssrSSSsS  zsaxSiJ^^ 

USSm  ^sii-ss'.rf'.Ji'JiErSaa  oE  ss  3sgcas£rns&&  mr^^rn  e£  caas  to  arsSS^ 
fssgalSgrffflia  cE  inr?  st  Sjzssoss,  it  •n^.'p^  ses^  eca&ssst£em  eE 

ccac^s,  gcn?!«;p  mra'gfHip'agtr^  Stn»  tpgfpvrg-  £s  gjjsajser:  0-5^  «£33.  cnC: 

slter  Csss  cgosSssSaQ;  e.g.,  3}>  aaMgSfcy,  25  t3r;>3?g?r  DeDer&ts, 

3J  -Sttost  5^3  233  4J  Sas  Eazsesst  oE  mfrSsa?-  caiSisSiars  rns^  gia  Sss  TTS»rrnfi»>rl 
Ew?  13  penessta  xsifira  sa  effect  jta  Sse  cscasScsfffia  to  ocg'  •»«?  =  0-S2), 

e&a  it  is  passible  to  oe:^  toe  pzcpcsitiso  qy  TtT^ 

ssd  csssEsssed  cy  tois  slztij  Ssst  tee  mndeir  colSlsica  preoSsm  wiH.  grew 
£=ri=s  Sbs  eezt  deesds  ss  egezstiesss  grew  if  sigufficszt  ssd  -zssa  jaetEods 
of  sir  traffic  ceotrol  are  sot  sdspted.  Is  tMs  case,  tsere  wos2^  se  a 
64  percent  re&gticq.  is  Sfe  galllar  benefits  clxised.  Figcre  30  ssews 
hots  cssasz  1}  midair  coUisiccs  are  assaaed  to  renaia  at  toe  lerel  of 
^  experienced  la  1958;  2)  colUsioas  are  fcmecast  to  grew  as  cossstiass 
Ssom  to  a  level  of  62  1977-  iSs  ccaclcsioa  xbat  toe  iastallatioa  of 

S3SS  III  ecaipiaeat  is  warranted  at  all  third  lot  isj  locations  resaias 
unchanged-  CKl^r  cosbiaatioos  of  chaises  in  tbe  data  can  he  estimated 


in  a  siai  lar  sanner- 


jeggtRnfiSs  3i 


amaT^tdEexl. 


SteieraB.  "feBSKC-sunnaEar*  faoeatrfszafcrjffl  tsEse  ctnnfccaDasfls 
afeSSg:  GgEssx&nosi.  &tia1^£s  soiS  laghrfffiffggg- 

1-  !*gtfgc  ssnesrisacsr  gvssissmnsS.  exsIssseS  tSis  ffirfggga  ggaartS  of 
ctnrsBfcs:  &sss.  Im  cvcSss:  t©  eairfegm  giheSr  fagcessSatrffi,  ccCaSasS  ttarac^ 
gesganffil  e&ssrscstfois,  cf  eSa  pslgarTO  ssSsgy  e£  cae  tnso  jg^aggaus,  &s 
gra2cr.eg3'  ssScaie  wes  sssSsseS  {£»  ggflVggg:  e5i£  serfctcsaess  of 

sU.  gsggcaiaa  -oriolsgfccs.  USe  rssaSrs  zre  ^sssssirssS  h&Isa. 

IS-  iSss  csnfirellsss  tras  ttcicfe  pane  Sn  see  ezsffirEnenr  tseae  g-eCTifftstfi 
arasra  s.  mdlsr^Is  g?Hng«ni»  e^-ge  of  <r”ig^rs«-«--i!:»Cr^  cfesfgnsd  S©  ejsSala  Saear 
sapjeesfae  coassaco  gsmerpragg  See  gelaSrise  megrsSrs  cf  See  griirs  sod  magirsl 
sgrsSEns  of  cffiga^ol-  He  resgossss  osgjlgg  wism  a  gaagsaasco  cf  See  esbo 
sysSsns  ia  Ei^aa3  to  ^sety,  easEfatSccs  imaiEneat:  cf  tsaffac  aad 
csctEclIsr  wscalcafis  «ss&  fcasd  to  gseatlj  fsxsc:  £S15.  lattia  esrgassis 

c£s  bsaa  placed  ca  frr>e<ga  aesaltSj  laoEiEr-  Ibsj  sEre  ccmpiled  ss 

1/ 

pert  of  a  steaderd  «^>rfefiEg;  prccedcre-  —  ^sidfes,  it  was  cleer^ 
endesstood  cefere  SZE  ccodcct  cf  See  ezperimaEt  taat  essaEssicos  of 
coctrollsr  satisfactloa  were  sot  scfficiect  to  Jcsti%  See  isstallatica 
of  ^.SIS  Ill  ecaipsEst.  SagErieaoa  wisii  tais  eqaipneat  at  tes  Atlanta 
aad  fork  facilities  iadiceted  that  the  ccotrollers  liked  the  ASIS 
SjStea  Just  fine. 

He  cosolete  telmletica  of  xespooses  to  all  questicsis  asked  is 
available  froa  ItSIEC  as  "Coatrolier  QiEstioisxaire,  **  project 
154-007-01X- 
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SIS.  S&s  S!:iSr£  Csgs'ugacrSam  gssfIrcaetB  ggagEsafsaiS  ameSyg!^  e£  '«as:±n&s 
ffigeg^ffgfca  taaifhftnnxsficBS,  reSjtt&na^gs  fa  or^fi&r  ezs  dfeceagfas  cBxsss  fafSaftfiiaiS, 
SgntxHUgs  cS.  r^SS  'ESS.  cihis  eairggfihicffifl  esess.  eo  vS&  sefissssS  gignBgas  e£  cxasr 
£ifsls  e&^sveS  xsf cMs  sfjfs&ssb  fa  ass.  S&sse  snal^S^s  ass  scssnEsSssS 
SsSsar. 


2®.  “Ifiia  'Erggffijtffi.uasfflgi  ^ngaas  pessanassS  a  cff  satstSsrScaS. 

gsfgg  rggrSS  gams  rg>?-ay-grTT}^BSg»s  Thrt*-  w£ta  S.  'Vf£9? 


craastfs  «?ig?np-n-5rpiT^  ^a&egrsr  afrcsafc  afesttS^Ssf  aacSi  aa  'Hinr  t5»T&< 
were  Sefasr  aaSe  ea  sr^  ocC:  eS  cstsfSfsas.  Tife^«s  Ss  fa  onarraas:  ca  pre- 
wfass  asaSfsaiS  tmS^  ■bcss  eaorssreS  cnHj  tsSsri  earas  igfsssin"s  eSaSSSy  ca  sxca£ 
cstrflfefts  scsr  all  aaressSfc,  USboe  C,  ficr  ezanpSs,  cn  sans  gfaea  sSrcssait: 
at^  gnace  a  cfflcC3©13£s"s  Ji»  easfer  sa  Sss£  ccSer  ssrtsrs^^  sifiract  Sras 
egrtSaness:,  oesefSfi:  Sa  sSe  easerff  cs  ajaaSjss  ecafiScts.  SEese  araSysss  are, 

'BgirtoCTaa;*^  ^'<!iiiuirhy.<a^  DeSjCRT- 

1.  ijasac  Cneraticggl  Eralcafiro 

•gSaa  aeS-ffliafs  scs  sieassri:^  s^siaSrSca  dissarees  were  eanplajedr  ISfeSiard  A 
csad  mimmnii  sSast  sarige  as  sre  tneassre  cf  ssparasacc-  ca^irjd  3  ased  sspsrs- 
tica  criteria  is  bsSa  vertical  ssS  bsrizcctal  directicQS.  Focx  lacaticcal 
categories,  I-I?,  -sere  ccssldered.  weiglitirg  scbame  enalogped  £or 
category  of  vlolatica  aaii  saSaiagca  separata oa  distaace  is  smss  ia  Table  A-1. 

.^^Ijizg  tiese  weigbts  to  the  record  of  ccaflicts  results  ia  the 
sismaiy  tahulatioa  sbova  iu  Table  A-‘2.  This  tabolatioo  confizas  the  con¬ 
clusions  provided  the  sore  objective  statistical  analyses  of  variance. 
Conflicts  vere  Judged  to  be  redxxred  is  both  vertical  sad  horizontal 
directions  for  the  certificated  and  trainee  controller  using  A3TS. 

Ihe  trainee  controller  benefitted  to  a  greater  extent. 


Tnz2>l!£  i&rl 


gsi^gsns  1SSS3S  m  gsgssnr  sgassaro 


BmmrEmm:mssc 


Igcagf  ffggiB.  Cgjggsgfes 

1-  Cta  IflffisiHizsr  ^iaS  e£  rarroigr  —  ffifcger  ncgdar^ 
IZ-  Cto  IggaBSaeg  ^(jjticsftSa  c£  erases:  nnssaer]^ 
m-  IfeatBC  fees  pgr£T>^  ncinc>  ZetaeJ 
1QT_  (jezee  ^zee 


fmrrg^Tngrff-«;  o£  SaZT  ta«^<?VrSr>rT»  Sfrt>grTn» 


HsgfJMTP-t-aTI 


a> 

S  ta  0.99  nT!g'g‘»«g 

b) 

Z.O  ta  "S^*^  tTp51I<sx: 

?  -^S  th™  2  Q5  irrS'afSKi; 

geigbts 

Iccaticoal 
CaSa^c-sy  I-H 


Locational 
Category  III-IV 


Teggical 

a)  0  tea  299  Seeg 
b>  33S  go  599  feet 
c)  6£0  tea  S5®  fees 


Harizaatal 


Eorizoatal 


SCHHICE:  "iseighted  Scores  on  Conflict  Data" 

Letter  D.  Brown  (KA~516)  to  S.  Horowita  (EC-100) 
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'H^>2£  A~2 

gSreagga  ^Rgs  sf  cssfliicss 


ESaSE 

aaicBg  2?  safbc 

sSssssl 

MrBC  Coctrollers 

Bs^azoetal  Sesaratioa 

1355 

2.922 

^tertical  S^aratioa 

2-.CC0 

3.211 

Field  gjSGfcgollers 

Bcsizcstal  Secaratica 

1355 

3.550 

Tertical  Senaratios 

1-722 

3.7S3 

SDESCSr  XbiiS.a  A-1- 


2.  Mi'-^^^  Sfca£is£isal  Aagiysis  cf  Cteflicts  Ttefra 

'!i:s  foUosiss  dascriptioas  sad  tasgalar  presesaati^gas  of  toese 
02552  beea  raprodaoed  from  tis  report. 

”4,3  5sgressica  Ketaodolcgy 

As  aa  iafossaal  part  of  tiie  project,  HilHE  agreed  to  rca  scae 
aoltiple  regressicHis  ia  the  area  of  safety  aad  iacez  of 
orderlii^s.  Tnrae  versions  of  index  of  orderliness  »rere 
supplied  by  mSEC,  ^d  the  assueption  is  sade,  for  the 
purposes  of  tl^e  regressions,  that  index  of  orderliness  is 
a  valid  oeasure  of  conflicts,  or  potential  aidair  collisions.^ 

Three  regressicns  vere  perforaed,- 

regression  #1 

I.O.  +  1  =  a  IFR  P  VFR^  Mode  ^ 
regression  #2 

1.0.  =  1=  a  iPR^  Mode  ^ 
regression  #3 

I.O.  +  1=0  UNK  ^ 


i 


ii 


i 

( 


i 

I 


1 


iSssrufi 


fl  ,  ,  iss&  tiae  r^isessiso  pgggneeers  &o  be  gsereEimared 

(aa£fgEeg£  far  regsessioa  fbs) 


I.O-  =  of  or^erlisess 

lEa  =  lEa  rzdsr  coegst 
¥3S  =  ¥5S  EaSsr  cccsS: 

SSoi^  =  &2ce:£oa  Sfisds  C  ecsi^ed  aircrafS: 

=  radar  cotst  of  all  trfeancn  aircraft  'tzasre  tssknasen  is  cefirgd 
as  follfsgsr 

ISfesEsal  -  all 

SSSS  III  -  all  acq-rnode  C  ‘C?! 


Bafa  for  tbe  regressions  tsas  arailaale  for  each  mamte  of  each 
nn.  Iha  r^gressisHS  vss  ran  separarely  for  manual  and  AHIS, 
cerrificated  and  noa-certificated;  £has  there  were  18  x  60  -  1030 
data  points  per  r^ression. 


All  three  regressions  sre  in  exponent  fom.  Ih^  be 
converted  to  a  lirear  fom  by  taking  logaritms  of  both  sides, 
as  follows: 


log  (I.O.  -5-  1)  =  log  Cl  -r  p  log  I5B.  T  Y  leg  VFR  -f  &  log  HODS 

lag  (I.O.  -rl)  =  loga^]$logIFR-fblog  KOBE 

log  (I.O.  -S-  1)  =  log  Cl  -r  0  log  05K 

The  logarithn  operation  demands  that  no  original  variable  take 
on  a  aero  valis.  It  is  for  this  reason  that  the  regressions 
are  vitii  I.O  -f  1  as  tte  dependent  variable,  since  I.O.  can 
assure  a  zero  value.  In  cases  where  the  independent  variables 
assured  zero  values,  a  snail  positive  valxie  vas  substituted. 

Standard  prograns  were  IBH*s  Scientific  Subroutine  package  (SSP) 
for  Systen  360  were  employed  to  effect  the  regressions.  The 
output  is  sisnaarized  in  Appendix  E." 

—  SOURCE:  "Results  of  HTTRE's  ARTS  III 
Vali,dation  Effort,"  Ibid 
Ref.  8. 


*The  form  of  the  regression  equations  were  specified  in  advance.  The 
first  of  these  versions  recorded  all  potential  conflicts  without  regard 
for  whether  violations  were  in  the  horizontal  or  vertical  direction. 

The  second  version  recorded  violations  in  both  directions,  but  included 
severe  violations  only.  The  third  version  was  similar  to  the  second,  but 
the  separation  standards  were  somewhat  relaxed. 
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3-  laterpretatioo  of  MUBE's  Re5q:ession  Statistics 

Conr^ents  sill  le  coofised  to  the  index  of  orderliness  version 
msz^rs  2  and  3  for  certificated  controllers.  Version  nusber  1  reflects 
slant  range  separaticai  only  for  ^ich  no  legal  standard  exists.  Astd 
in  keepii^  in.th  our  previous  discussions,  the  interpretation  of  trainee 
perforaance  will  be  United  solely  to  inferences  about  the  ability  to 
train  cratr oilers. 

Regression  nuaber  1  tells  us  we  can  infer  that  the  number  of  » 

conflicts  gro5?  at  a  rate  idiich  is  less  than  the  square  of  the  traffic 
(the  rate  to  expect  if  a  "gas  molecule"  collision  model  were  appropriate) . 

The  exponent  for  IFR  traffic  ranges  from  0.6  to  1.0,.  These  values  are 
statistically  significant;  they  exceed  their  standard  deviation  by  more 
than  a  factor  of  two. 

However,  the  exponent  to  the  level  of  VFR  traffic  does  not  differ 
statistically  from  zero.  This  result  was  not  unexpected  since  the 
experimental  design  employed  in  the  NAFEC  simulation  did  not  have 
sufficient  variation  in  VFR  traffic  to  estimate  a  reliable  coefficient 
for  this  variable.  VFR  traffic  was  approximately  constant  in  the 
experiment,  and  the  statistical  analysis  confirms  this. 

The  exponent  to  the  proportion  of  Mode  C  behaves  according  to 
expectation  in  that  it  is  a  negative  value ;  the  higher  the  proportion 

r 

of  Mode  C,  the  fewer  the  conflicts.  But,  there  is  a  minor  anomaly 
in  that  one  would  expect  this  relation  to  hold  only  for  the  ARTS  III 


Regression  #1 


I.O  +  1  =  a  IFR^  VFrT  Mode  ^ 


Version 

Runs 

a. 

r.c. 

r.c.. 

s.d. 

Y 

r.c. 

s.d. 

s 

r.c. 

S.d. 

Correlation  Coefficients 

IFR 

VFR 

Mode 

Multiple 

1 

MAN  -  CER 

.304 

.574 

.070 

-.016 

.021 

-.413 

.202 

t 

’  .28 

.15 

-.02 

.29 

ARTS-  CER 

.316 

.508 

.062 

-.011 

.019 

-.732 

.187 

.28 

.15 

-.08 

.30 

MAN  -NCER 

.342 

.557 

.079 

-.026 

.023 

-.399 

.217 

.23 

.12 

-.03 

.24 

ARTS-NCER 

.268 

.624 

.067 

-.050 

.020 

-.601 

.195 

.28 

,.11 

-.06  • 

.30 

2 

KAN  -  CER- 

.144 

;  1.022 

.104 

-.G43 

.031 

-.801 

.301 

.32 

.16 

-.04 

.33 

ARTS-  CER 

.156 

.916 

.096 

-.012 

.029 

-1.301 

.290 

.32 

.18 

-.09 

.34 

MAN  -NCER 

.218 

.889 

.112 

-.039 

.033 

-.814 

.305 

.26 

.13 

-.05 

.28 

ARTS-NCER 

.130 

1.131 

.100 

-.128 

.030 

-.727 

.290 

.31 

.09 

,-.05 

.34 

3 

MAN  -  CER 

.233 

.782 

.084 

-.043 

.025 

-.317 

.241 

.30 

.14 

-.001 

.31 

ARTS-  CER 

.308 

.626 

.076 

-.013 

.023 

-.748 

.228 

.28 

.16 

-.06 

.30 

MAN  -NCER 

.403 

.591 

,  .094 

‘-.023 

.028 

-.453 

-256 

.22 

.11 

02 

.22 

ARTS-NCER 

.251 

.763 

.079 

-.072 

.024 

-.566 

-230 

.28 

.10 

-.05 

.30 

system.  Logically,  Mode  C  is  a  silent  contributor  to  the  manual  system. 
It  should  have  no  effect  on  conflicts.  Version  3  behaves  nicely  in  that 

t 

thu’ exponent  for  this  variable  does  not  differ  from  zero  for  the  manual 
system,  whereas  it  is.. a  negative  value  for  ARTS.  Version  ,2,  however, 
shows  a  negative  value  for  both  systems.  This  is  due,  probably,  to  the 
fact  that  the  equation  forms  used  in  regression  1  could  not  differentiate 
between  specific  transponder  features;  an  aircraft  with  a  Mode  C 
transponder  x^as  still  able  to  provide  a  double  slash  symbology  when  the 
manual  system  was  being  used.  Therefore,  the  greater  the  proportion- of 
Mode  C  equipped  aircraft,  the  fewer  the  conflicts;  even  for  the  manual 
system.  ‘Hoxirever,  in  all  versions  for  the  index  of  orderliness,  the 
negative  value  is  larger  for  the  ARTS  system,  as  it  should  be. 

Regression  number  2,  deletes  VFR  traffic  as  an  explanatory  variable, 
since  it  was  expected  that  its  effect  would  not  be  significant.  There  is 
little  change  in  the  results.  The  exponent  to  IFR  is  about  at  the  same 
level;  a  range  from  0.6  to  0.9.  Both  sets  of  equations  show  a  lower 
growth  rate  in  conflicts  using  ARTS.  The  exponent  to  the  proportion  of 
Mode  c  transponders  shox^s  the  same  pattern  uescribed  previously  in 
regression  number  1. 

Regression  3,  attempted  to  pinpoint  the  contribution  made  by 
Mode  C;  to  differentiate  between  double  slash  identity  available  from 
the  manual  system  and  the  actual  reporting  of  altitude  information.  It 
should  be  noted  that  the  growth  rate  for  conflicts  as  the  number  of 
"unknown"  aircraft  grows  is  less  for  .\RTS  than  manual.  Moreover,  the 
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FIGURE  A.1 
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difference  in  tc=  raSre  o>£  grostfi  setseee  SS3S  sd  ccsssal  is  sfcaSistical^ 
sigcificest.  Figure  A-1  is  a  gragMcai  psesesEatioa  of  tMs  resoiC- 
Tiiis  resalfc  vould  have  ismess^e  practical  sigaificaace  if  esplsaatory  power 
of  the  regression  equation  were  greater,  lisis  poser  is  defined  els: 
coefficier^  of  correlation;  the  square  cf  tisis  coefficient  represents 
the  percent  of  the  total  variation  in  tbz  data  is  accounted  for  by 
the  single  variable:  “ua&aova  aircraft."  This  percent  is  low  for 
regression  nuebers  1  and  2  (13  percent  of  tis  variance  is  eanlaited), 
and  even  lower  for  regression  msiier  3  (2-3  percent  cf  tbs  variance  is 
^counted  for).  But,  correlation  analysis  tells  us  that  this  percent^© 
contribution  by  Mode  C  is  highly  significant  statistically;  the 
coefficient  of  cecralation  is  five  tines  as  large  as  one  would  erpact 
fron  a  randon  sanple  of  the  size  used  in  this  analysis  (1030  data 
points).  But,  the  causes  of  conflicts,  as  revealed  by  the  correlation 
analyses  described  above,  are  so  varied  that  even  though  ve  can  clain 
that,  statistically  speaking,  the  number  of  unknown  aircraft  explains 
the  occurrence  of  conflicts  to  a  significant  degree,  as  a  practical 
matter  they  do  not  explain  very  much  (97  percent  of  the  variation  in 
conflict  data  remains  unexplained). 

4.  TSC  Statistical  Analysis  of  Conflict  Data 

A  parallel  effort  was  undertaken  by  TSC  to  determine  the  isolated 
contribution  made  by  the  addition  of  Mode  C  capability.  This  effort  was 
concerned  primarily  with  an  examination  of  those  aircraft  which  came 
into  conflict.  Sophisticated  mathematical  techniques  were  employed  to 
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£££  Ks>5s  C  ard  fiaafffc  3e=±£mtt^  xrx=±E&3£s  ea  gSts  ateSistl  gatassd  e£ 
oscflleSs:  Sgtogg  can&s:lic£ss  «£s  sn(t  csied  ss  £  ga'axy. 

la  gggegala  cMs  effiarfc  gseiseeffid  refgal&s  ucicaa  'sese  isssoclssi'se. 
TSC  (Sid  eesefizssj,  hsasips^^  graS:  "152  traffic  &ssi£gf  is  psssiessfiSy 
carcelsted  tsrStn  C(33£iic£s  asd  cercssr  3fiB(5£  C^5SJ  5S 

correlated  witii  cccflicts."  It  was  Sscsd,  for  eianpis,  fr'g’cgy  “isb,  traffic 
deasity  ffmucrg*ts  Sar  t&e  lica”s  ^ssre"  of  tos  coafSicts,  ccS:  Sams 

to  lease  taraassered  tre  caestioa  as  to  wfetfer  a  feaswled^  of  t£e  112 
traffic  desssitj  coatrisctes  sigai  ficaatly  to  tae  aroo^ilft^  of  coaflict 
waea  na  traffic  dsasity  is  featica."  Farciser,  ISC  foessd  S±at 
increased  safetj  for  Mods  C  aircraft  was  sisosn  by  tress  stcdies.” 

TSC  coaclxdes  that,  ”ocr  chief  accocgl isroeat  was  daralopii^  tias 
aaaljFsis  package,  gainirg  erperieace  ia  csisg  it  ssS  discovarisg  its 
weak  poists."  fault,  however,  was  aot  ia  tiaear  aethod  or  lack  ctf 
desire.  It  was  sii^ly  a  oatter  of  trying  to  sqieeze  sis^rs  beycad  tteir 
usefulisss.  The  stu^  was  not  designed  to  provide  corclcsive  iaforaatioa 
regarding  what  specific  feature  of  £S3!S  III  was  Eaost  useful;  Mode  C, 
ground  speed,  alphanumeric  identity,  etc.  It  was  intended  only  to 
coopare  the  package  of  ARTS  III  automation  with  the  sanual  ^stea  of 
air  traffic  control.  The  follow-on  analyses  revealed  that  a  newly 
devised  experiment,  not  finrther  manipulation  of  the  conflict  data,  is 
required  to  answer  more  specific  questions  concerning  ARTS  III. 

In  summary,  while  the  inability  to  separate  the  contribution  made 
by  individual  ARTS  III  featinres  is  a  disappointing  result  of  the  total 
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fisHsssP-on  aaai^itricsiS.  <e£Sggc,  aaara  ^aerffi  a  fas?  nnrsSfist  2;2i2»  sffiaestSffilagtg- 
iSsiE  r^atst  of  ssssfs^  Sa  carflffiSs  as  a  fcnffiSrsara  of  gzaSfge  gsasSsy, 

«ss  reraaSisS  is  ss^Sjsis  ttso  fscsa  aa  ezsoossSdEal  aase 

of  gccac^  of  l.@  £ssr  gfefc  ttiganaiS.  gg^SEa  fa  r^cefafoa  2  (raeosS  of 
£e??£r£  'vfolatiocs),  screes  gaefe  gns  ass^sSs  cf  rear  imSss  fess 
in  gig^re  9.  Esfia  sears  of  daaa,  t^gafaafl  fEuun  iCTSage-rroernt  sscrcss, 
sjagjffl're:  tea  caatee^iica  of  &  gsaporticralj  aatrer  fran  s  sscsaeed  relatjcn 

ir 

to  traffic  actwrifiy- 

“Ifg:  <Ss£a  at  a  «TTt»r»S;TTrTn^  Ss  taet  zafcte  esctestioa  Cfat  SSsn^  C  is 
a  sigaificaai:  coafcrifctor  Co  trie  gR&ir.Hca  ia  ccssflicts.  “gfrey  iaficate 
tlsat  Kofe  C  alcce  coes  act  acccecrt  fsr  gfcis  ce^txctioa.  SSorecKiar,  all 
regressioa  asaljses  larfossed  hj  Ifa  Kiilifeg  Corporation  reraal  taat  tne 
rate  of  grosta  of  cossflicts  with  r^arc  to  traffic  activity  is  less  for 
the  5B3S  SjStea  than  sraagal.  "Ims  resait,  althocgh  statistically 
sigaificaat,  indicates  that  a  aore  precisely  ccatrolled  ezperiaaaC, 
desig:^  for  the  single  purpose  of  esCiaating  traffic  activity  aod 
Hode  C  effects  is  required  before  practical  jndgsnts  can  be  made. 
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&.  ssm^  wfrtCT  rearsrffis  t&stz  >&£;  rggeSsa;  os  s  gSnpsfcgl 

fl»rrpg»^p<n»«ywr  ^  JisSsiK?  nfc&  &&£  agcxESoESssx  a£  X.  exs^l^cessSitL 
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